-- Logs begin at Thu 2022-09-01 17:10:37 UTC, end at Thu 2022-09-01 18:35:34 UTC. -- Sep 01 17:43:41.263267 dv-cinder-749155-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 17:43:41.271162 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 17:43:41 2022] *** Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: nodename: dv-cinder-749155-10-iscsi Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: machine: x86_64 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: clock source: unix Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: pcre jit disabled Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: detected number of CPU cores: 4 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: current working directory: / Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: detected binary path: /usr/bin/uwsgi-core Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: your processes number limit is 47680 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: your memory page size is 4096 bytes Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: detected max file descriptor number: 2048 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: lock engine: pthread robust mutexes Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: thunder lock: enabled Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 17:43:41.274764 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 17:43:41.314420 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: Python main interpreter initialized at 0x55f91c79d9c0 Sep 01 17:43:41.314420 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: python threads support enabled Sep 01 17:43:41.314420 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: your server socket listen backlog is limited to 100 connections Sep 01 17:43:41.314420 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: your mercy for graceful operations on workers is 90 seconds Sep 01 17:43:41.315032 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 17:43:41.315032 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: *** Operational MODE: preforking *** Sep 01 17:43:41.315174 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: *** uWSGI is running in multiple interpreter mode *** Sep 01 17:43:41.315174 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: spawned uWSGI master process (pid: 81690) Sep 01 17:43:41.315366 dv-cinder-749155-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 17:43:41.317178 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: spawned uWSGI worker 1 (pid: 81703, cores: 1) Sep 01 17:43:41.317178 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: spawned uWSGI worker 2 (pid: 81704, cores: 1) Sep 01 17:43:41.317693 dv-cinder-749155-10-iscsi devstack@keystone.service[81690]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 17:43:45.770702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:43:45.770702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: from cryptography.utils import int_from_bytes Sep 01 17:43:45.771492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:43:45.771492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: from cryptography.utils import int_from_bytes Sep 01 17:43:45.774599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:43:45.774599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: from cryptography.utils import int_from_bytes Sep 01 17:43:45.775089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 17:43:45.775089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: from cryptography.utils import int_from_bytes Sep 01 17:43:45.823151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 17:43:45.823324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 17:43:45.823517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] command line args: [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ================================================================================ {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] admin_token = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] config_dir = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] config_source = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.827253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] debug = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.827714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.827968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 17:43:45.827968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 17:43:45.828087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] command line args: [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 17:43:45.828212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 17:43:45.828372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ================================================================================ {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 17:43:45.828495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] admin_token = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] config_dir = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_config_append = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.828944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] config_source = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_dir = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] debug = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_file = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.829954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_config_append = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_dir = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_file = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.830988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] publish_errors = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.831926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] standard_threads = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] publish_errors = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] use_journal = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.833941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] use_json = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] use_stderr = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] use_syslog = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.834637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] standard_threads = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.834793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.834864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.834934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.835009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.835080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.835150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.835296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] use_journal = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.835296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] use_json = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.835433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.835514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] use_stderr = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.835587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.835662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] use_syslog = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.835809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.835897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 17:43:45.836061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.836890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.837917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.838968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.839919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.840956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.841909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.842849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.843991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.844931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.845867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.846941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.847945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.848109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.848189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.848361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.848442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.849896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.850822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.851966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.852824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.853935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.854972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.855983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.856940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.857971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.858925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.external = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.859963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.password = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.860932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] auth.token = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.external = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.861928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.password = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] auth.token = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.862928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.863877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.864046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.864132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.864854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.864945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.865851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.866963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.867944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.868142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.868246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.873953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.874971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.875882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.876939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877124 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.877939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.878982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.879062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.879226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.879464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.879544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.879705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.880044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.880218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.880296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.880522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.880614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.880754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.881953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882495 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.882957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.883156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.883345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.883513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.883762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.883832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.883998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.884232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.884357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.884494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.884717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.884797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.884954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.885172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.885725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.885794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.885984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.886065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.886416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.886495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.886686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.886767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.886980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.887970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.888951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.889038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.889179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.889257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.889417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.889499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.889658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.890948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.891976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.892853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.893969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.894913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] role.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] role.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.895871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.896984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897687 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.897998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.898892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.899998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.900994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.901934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.902908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] role.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.903862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] role.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.904861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.905851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.906963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.907858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.connection = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.908888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.909996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.910939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.911018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.911713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.911795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.911958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.912060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.912197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.912306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.917742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.917951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.918612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.connection = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.919733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.920231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.920231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.920231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.920470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.920679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.920916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 17:43:45.921082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 17:43:45.943302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.943532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.943755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.946261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.947927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.947927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.952190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.955292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.981111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.981337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.983382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:45.983382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.045480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.045858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.098155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.098385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.100689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.100926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.261930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.262014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.262142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.262225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.262351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.293917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.294889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.295018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.295109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.295231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.295314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.295442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.306490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.306869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.319357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.319479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.319586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.319672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.319813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.319915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.320849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 17:43:46.337933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.338145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:43:46.338382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.338497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.338583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:43:46.338697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:43:46.338917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.339010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.339118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 17:43:46.339202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.339467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.339643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.339724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.339914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.340181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.340479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.350020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.350300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.350573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.350857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.351470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.351749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.352019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.352297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.357857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.359250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.360766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.361054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.361342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.361646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.361913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.362155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.362238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.362432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.362522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.362710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.362793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.362986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.363068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.363263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.363345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.363522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.363602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.363905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.364210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.364517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.364801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.365068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.386422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.387565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.387896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.388181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.389103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.389381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.392998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.393289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.393580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.394281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.394565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.394847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.400838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.401129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.401701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.401983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.405083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.405381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.405668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.406013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.406108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.406229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.406350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.406628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.406911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.407185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.407460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.407728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.407996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.408147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.408464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.408754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.424715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.425002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.425612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.425890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.428850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.430008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.431942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.432238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.433122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.433416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.436609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.437669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.439676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.440927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.443261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.446798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.461066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.461360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.461633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.461902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.462895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.463185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.463456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.463727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.468742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.470411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.470708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.471953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.472235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.472339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.473833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.474112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.474936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.475229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.475524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.475817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.476109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.476891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.480844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.480844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.480844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.480844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.487367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.487647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.487930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.488211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.488531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.488814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.489109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.489242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.489526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.489904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.490173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.490463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.490742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.491052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.504604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.507191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.508338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.513903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.514001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.514001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.514143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.514218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.514446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.514544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.515006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.516262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.516576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.516876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.522434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.523921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.524066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.525694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.526826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.527131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.527421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.527702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.528578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.528879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.529164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.529436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.535629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.535923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.536392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.537476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.537763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.538040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.543319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.543616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.543903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.544183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.544486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.544771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.544855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 17:43:46.545052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.545972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.546976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.547107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.547389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.547663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.547922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.548186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.548496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 17:43:46.581267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f91c79d9c0 pid: 81703 (default app) Sep 01 17:43:46.583021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f91c79d9c0 pid: 81704 (default app) Sep 01 17:43:46.585492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6602a-7c60-4b53-8a96-9ac31c464d3f None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:46.585783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6602a-7c60-4b53-8a96-9ac31c464d3f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:46.586057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6602a-7c60-4b53-8a96-9ac31c464d3f None None] PATH_INFO: `/v3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:46.590268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Thu Sep 1 17:43:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:43:54.461376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.461931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.462097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.466647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:43:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:43:54.471931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:54.472494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:54.472494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:54.546392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: INFO dbcounter [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] Registered counter for database keystone Sep 01 17:43:54.567776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG oslo_db.sqlalchemy.engines [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81703) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 17:43:54.574393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writer thread running {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 17:43:54.769450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG passlib.handlers.bcrypt [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81703) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 17:43:54.769841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG passlib.handlers.bcrypt [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81703) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 17:43:54.828488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:54.830053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:54.964021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:43:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 17:43:55.420214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:55.490789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: INFO dbcounter [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] Registered counter for database keystone Sep 01 17:43:55.509413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG oslo_db.sqlalchemy.engines [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81704) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 17:43:55.515898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writer thread running {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 17:43:55.637448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a617db4-1396-428c-a631-ca19b3b3eb50 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:55.642038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:55.642662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:55.642949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:55.643228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:55.651344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:55.703301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 17:43:55.703653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754938f-cdfa-4b74-b065-8b0da4c15559 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:55.713156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Thu Sep 1 17:43:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 17:43:55.719093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:55.733900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebfef5-c449-44d1-8cb2-bc1a8210c783 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:55.738297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:55.739385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:55.739666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:55.739935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] PATH_INFO: `/v3/projects/195190be58ff49f7a586c1baca5891d6` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:55.741545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] RBAC: Authorizing `identity:get_project(project_id=195190be58ff49f7a586c1baca5891d6)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:55.795626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING keystone.common.rbac_enforcer.enforcer [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 17:43:55.795912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b8a7d-82b3-44bc-8f51-213732289a78 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:55.800491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Thu Sep 1 17:43:55 2022] GET /identity/v3/projects/195190be58ff49f7a586c1baca5891d6 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:43:57.684552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c959fe5-22e1-404c-b028-f5076a83dd11 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:57.685167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c959fe5-22e1-404c-b028-f5076a83dd11 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:57.685167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c959fe5-22e1-404c-b028-f5076a83dd11 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:57.689228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:43:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:43:57.695036 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:57.695460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:57.695624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:57.716333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:43:57.717873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:43:57.725716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:43:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 17:43:58.001690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c959fe5-22e1-404c-b028-f5076a83dd11 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.014337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c959fe5-22e1-404c-b028-f5076a83dd11 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.018868 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c03720-2f4f-4798-8f37-88cdcb4736a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.019469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c03720-2f4f-4798-8f37-88cdcb4736a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.019749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c03720-2f4f-4798-8f37-88cdcb4736a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.020023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c03720-2f4f-4798-8f37-88cdcb4736a1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.028223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c03720-2f4f-4798-8f37-88cdcb4736a1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.030020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c03720-2f4f-4798-8f37-88cdcb4736a1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.038879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Thu Sep 1 17:43:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:43:58.044249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:43:58.054494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f53ad-850f-4de0-a952-86e32d24e390 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:43:58.058877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5e8b7-635b-4844-8c64-fe6fbdf7b9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:43:58.059512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e8b7-635b-4844-8c64-fe6fbdf7b9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:43:58.059799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e8b7-635b-4844-8c64-fe6fbdf7b9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:43:58.060078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e8b7-635b-4844-8c64-fe6fbdf7b9f1 admin admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:43:58.061440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5e8b7-635b-4844-8c64-fe6fbdf7b9f1 admin admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:43:58.063261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5e8b7-635b-4844-8c64-fe6fbdf7b9f1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:43:58.064826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Thu Sep 1 17:43:58 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:00.027643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463acafb-94da-43bc-ac44-a7454bfc42c3 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.028193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463acafb-94da-43bc-ac44-a7454bfc42c3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.028193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463acafb-94da-43bc-ac44-a7454bfc42c3 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.029541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:00.034698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.035111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.035258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.054321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:00.055485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:00.104886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a2b4-ba5b-415f-97c7-d32734a0eb5b None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.105173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a2b4-ba5b-415f-97c7-d32734a0eb5b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.105451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a2b4-ba5b-415f-97c7-d32734a0eb5b None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.106782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:00.111818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.112102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.112403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.135621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:00.154069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG passlib.handlers.bcrypt [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81704) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 17:44:00.154454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG passlib.handlers.bcrypt [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81704) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 17:44:00.203871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:00.205454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:00.311319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:00.406250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:00.430667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:00.510309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8f8ae-bcbf-4b27-9e33-41fc8c282319 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:00.544370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:00.545003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.545286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.545560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.556299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:00.565118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:00.566197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:00.567245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:00.572372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:00.590112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:00.678286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:00.684305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26eb09-177a-4045-8c98-d5454c241124 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:00.686696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:00.696338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504278b-e04b-41e9-9775-9a9e09030c18 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:00.722783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:00.723715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.724003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.724446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.725306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:00.725609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:00.727030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:00.730950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:00.731560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.731844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.732117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.738591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:00.744260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:00.800960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:00.803009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:00.847213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0b2c-960f-4ced-820c-75c96ef2e3fa None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:00.881471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:00.882322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:00.882541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:00.882662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:00.882861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:00.884498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:00.885943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:00.887511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 9/16] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:00 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:00.893296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:00.900924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 8/17] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:44:00.906418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:00.927086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:01.002199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6a9af-c7c2-4bd4-964a-a71da6544231 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:01.027837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff8d28-15ab-40c1-b4d4-cd906e241a71 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:01.036973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:01.038117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:01.038410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:01.038682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:01.039826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:01.041303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:01.042869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:01.048802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:01.063076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:01.063717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:01.063993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:01.064260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:01.074084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:01.075551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:44:01.093002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:01.093915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:44:01.094825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:44:01.094825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:44:01.094825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:01.094825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:01.100346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:01.133849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:01.149429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91533aaf-fc01-472a-8363-36299b5e56ec None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:01.150983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:01.184150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:01.184793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:01.185075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:01.185350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:01.186080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.domain.id=default, user.id=1369adf90e6146c298833c5db7ff437b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:01.186386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:01.187816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947e1b6-4448-4bea-9b56-b11cf6237b8f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:01.196437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Thu Sep 1 17:44:01 2022] GET /identity/v3/role_assignments?user.id=1369adf90e6146c298833c5db7ff437b&scope.domain.id=default&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:01.199485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87672e8e-4935-4f03-9bb0-067c5a2f70da None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:01.237660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:01.238372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:01.238653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:01.238927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:01.239597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:01.239930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:01.241412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60a7b6-8557-44d6-9c2b-1bfd8229e990 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:01.261817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:03.338259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c8e96-7ae8-449f-9d17-da465a851ffb None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:03.339222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c8e96-7ae8-449f-9d17-da465a851ffb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:03.339689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c8e96-7ae8-449f-9d17-da465a851ffb None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:03.341559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:03.348077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a1c76-f9b5-4142-a4fe-5a86207d47c2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:03.348983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a1c76-f9b5-4142-a4fe-5a86207d47c2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:03.349689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a1c76-f9b5-4142-a4fe-5a86207d47c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:03.376666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-597a1c76-f9b5-4142-a4fe-5a86207d47c2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:03.378209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-597a1c76-f9b5-4142-a4fe-5a86207d47c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:03.482581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:03.719933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1323b0d-6681-42bf-be47-86fb5013f8fd None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:03.720651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1323b0d-6681-42bf-be47-86fb5013f8fd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:03.721153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1323b0d-6681-42bf-be47-86fb5013f8fd None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:03.722699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:03.728866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8173ee-1c0b-467d-8fb3-f576121da7ca None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:03.729416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8173ee-1c0b-467d-8fb3-f576121da7ca None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:03.729910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8173ee-1c0b-467d-8fb3-f576121da7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:03.754005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ae8173ee-1c0b-467d-8fb3-f576121da7ca None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:03.755530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ae8173ee-1c0b-467d-8fb3-f576121da7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:03.778449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:03.841291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 12/25] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:04.068075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1323b0d-6681-42bf-be47-86fb5013f8fd None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.182468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1323b0d-6681-42bf-be47-86fb5013f8fd None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.217455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.218450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.218946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.219430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.223532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236548d6-0cba-4004-bca5-9a5bbd8a15e5 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.224657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236548d6-0cba-4004-bca5-9a5bbd8a15e5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.225148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236548d6-0cba-4004-bca5-9a5bbd8a15e5 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.227239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 13/26] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:04.233259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.233554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.233815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.243075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:04.252375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:04.259036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:04.260861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 14/27] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:04.262722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.290105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:04.291757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:04.335404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:04.410584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 14/28] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:04.411684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.422690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb4938-d115-465d-8d28-d2b62fc7cfa5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.463112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.464445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.464887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.465314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.466541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:04.467263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:04.469057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ead5-d1af-4a35-aa65-d616b0679ecb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:04.480190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 15/29] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:04.482098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9017af-bddd-4fd9-89c1-74750fd8a37d None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.482703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9017af-bddd-4fd9-89c1-74750fd8a37d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.483114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9017af-bddd-4fd9-89c1-74750fd8a37d None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.484572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 16/30] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:04.488364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9017af-bddd-4fd9-89c1-74750fd8a37d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.524102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:04.556418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd906c2-bcd0-4a52-9ebb-fe03a535f7ae None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.624655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.625296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.625723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.626135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.632407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:04.645574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:04.647458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9017af-bddd-4fd9-89c1-74750fd8a37d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.648000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-629a768e-357d-4133-9339-cd0c6eb5586e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:44:04.680601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:04.681561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 15/31] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.683455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.699111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.699602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.700625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.703001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:04.705684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:04.709797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 17/32] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:04.711566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.728397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:04.729988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:04.842580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 16/33] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:04.843434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.869500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b683e8e-6239-45b0-a33b-bc1871490004 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.907231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.908212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.908738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.909317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.910226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:04.910732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:04.912278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:04.912594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:04.935925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 18/34] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:04.937167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:04.949463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7103d2-f2ce-409a-8b7b-3c586331cd4c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:04.960875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:04.985590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:04.986226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:04.986508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:04.986780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:04.987892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:04.989360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:04.991612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 17/35] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:04.996439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:05.093597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370f107-2e40-4ac6-83d9-54a1ac69baa3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:05.100788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036b6c-9f99-4ba9-b83d-cef68bec0295 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:05.156345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4232a-274a-467b-9e84-a348ed97eff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:05.156962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4232a-274a-467b-9e84-a348ed97eff0 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.157296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4232a-274a-467b-9e84-a348ed97eff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.157510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4232a-274a-467b-9e84-a348ed97eff0 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.158159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4232a-274a-467b-9e84-a348ed97eff0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:05.159595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4232a-274a-467b-9e84-a348ed97eff0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:05.164712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:05.165322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.165600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.165870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.166546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:05.167903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:05.175584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:05.384657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 19/36] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 1 17:44:04 2022] POST /identity/v3/roles => generated 219 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:05.385862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:05.416425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 18/37] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:44:04 2022] POST /identity/v3/projects => generated 301 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:05.417999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bf4e9-b399-496c-a806-bde293b88518 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.418294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bf4e9-b399-496c-a806-bde293b88518 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.418571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bf4e9-b399-496c-a806-bde293b88518 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.420189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 19/38] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:05.425440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bf4e9-b399-496c-a806-bde293b88518 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:05.532008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:05.561974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e2e5-280e-4fce-97b4-2026824e6ed1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:05.610966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7255ff-b890-4a55-a9f7-8d75bcfdcc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:05.611674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7255ff-b890-4a55-a9f7-8d75bcfdcc45 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.611957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7255ff-b890-4a55-a9f7-8d75bcfdcc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.612234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7255ff-b890-4a55-a9f7-8d75bcfdcc45 None admin] PATH_INFO: `/v3/domains/default/users/1369adf90e6146c298833c5db7ff437b/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.615749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7255ff-b890-4a55-a9f7-8d75bcfdcc45 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1369adf90e6146c298833c5db7ff437b, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:05.617404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7255ff-b890-4a55-a9f7-8d75bcfdcc45 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:05.681006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bf4e9-b399-496c-a806-bde293b88518 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:05.713216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 20/39] 10.222.0.15 () {58 vars in 1361 bytes} [Thu Sep 1 17:44:05 2022] PUT /identity/v3/domains/default/users/1369adf90e6146c298833c5db7ff437b/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:05.714614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593257-f433-4a8e-8b20-9b212265a644 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.714903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593257-f433-4a8e-8b20-9b212265a644 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.715182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f593257-f433-4a8e-8b20-9b212265a644 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.716850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 21/40] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:05.721268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.721605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.721895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.729098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e00569-6341-4f25-85f6-913735245059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:05.729754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e00569-6341-4f25-85f6-913735245059 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.730087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e00569-6341-4f25-85f6-913735245059 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.730369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e00569-6341-4f25-85f6-913735245059 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.731086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e00569-6341-4f25-85f6-913735245059 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:05.732737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e00569-6341-4f25-85f6-913735245059 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:05.748636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:05.758004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:05.820363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 20/41] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 1 17:44:05 2022] POST /identity/v3/roles => generated 225 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:05.821379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:05.821800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:05.822191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:05.832386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 22/42] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:05.848473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:05.851251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:05.908764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:05.937380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:06.112867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:06.180040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:06.216422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21e8b5-2ccd-4d83-bf96-66cb0468cbec None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:06.216580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:06.252272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:06.252945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:06.253531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:06.253531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:06.254602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:06.257032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:06.258890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 23/44] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:06.264218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:06.335920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd1c83-1af9-4840-90bf-834dc8633e20 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:06.373715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c06ae-f2de-4f55-8618-4d5fbef23464 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:06.374124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ff395-e203-45c7-914d-77481e584749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:06.374741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ff395-e203-45c7-914d-77481e584749 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:06.375024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ff395-e203-45c7-914d-77481e584749 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:06.375303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ff395-e203-45c7-914d-77481e584749 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:06.375985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ff395-e203-45c7-914d-77481e584749 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:06.377403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ff395-e203-45c7-914d-77481e584749 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:06.408221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fd3fd-a908-4c0e-8fb1-15e983ea4aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:06.408869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd3fd-a908-4c0e-8fb1-15e983ea4aab None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:06.409156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd3fd-a908-4c0e-8fb1-15e983ea4aab None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:06.409433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fd3fd-a908-4c0e-8fb1-15e983ea4aab None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:06.410085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fd3fd-a908-4c0e-8fb1-15e983ea4aab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:06.411551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fd3fd-a908-4c0e-8fb1-15e983ea4aab None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:06.416579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:06.499809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 22/45] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 1 17:44:06 2022] POST /identity/v3/roles => generated 223 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:06.536344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 24/46] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:44:06 2022] POST /identity/v3/projects => generated 312 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:07.700380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bb713-0955-4280-9f51-a0eb394af518 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:07.700926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bb713-0955-4280-9f51-a0eb394af518 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:07.700926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bb713-0955-4280-9f51-a0eb394af518 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:07.702293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 23/47] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:07.707542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:07.707929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:07.708111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:07.727020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:07.728226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:07.794608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 25/48] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:08.072962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bb713-0955-4280-9f51-a0eb394af518 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:08.109923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:08.178098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bb713-0955-4280-9f51-a0eb394af518 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:08.212092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:08.212752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.213039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.213316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.223412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:08.228993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:08.229843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:08.230623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 24/49] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:08.235924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:08.336200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0caebb-258e-4416-9488-c4434a658a20 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:08.352702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:08.371041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:08.371664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.371948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.372224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.372932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:08.373238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:08.374618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d85ef-8906-4c0a-94db-5f1e56beab22 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:08.383253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 26/50] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:08.389409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:08.445390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60181b8-eff4-4e42-91eb-390a5004a1ea None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.445708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60181b8-eff4-4e42-91eb-390a5004a1ea None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.445954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60181b8-eff4-4e42-91eb-390a5004a1ea None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.447294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 27/51] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:08.452164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.452498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.452785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.471093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:08.472302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:08.485935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:08.492514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e389618-d390-4f72-bb13-9ae31366edb7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:08.526869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:08.527944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.528229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.528562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.529698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:08.531251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:08.532871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 25/52] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:08 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:08.538993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:08.541241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 28/53] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:08.640878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1a0c-e618-4264-baa6-fe767b0384c7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:08.678259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:08.678885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.679167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.679441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.680582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:08.682130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:08.684265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 26/54] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:08.690913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:08.787396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3d554-c7cd-4749-8fad-b9e9e5f33e5d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:08.807943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:08.841796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:08.901658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:08.915863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c21a8-21fc-49b9-aeea-88a6aa2b7462 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:08.926503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:08.927155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.927435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.927882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.928649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.domain.id=default, user.id=1369adf90e6146c298833c5db7ff437b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:08.928958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:08.930465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:08.936259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 29/55] 10.222.0.15 () {56 vars in 1322 bytes} [Thu Sep 1 17:44:08 2022] GET /identity/v3/role_assignments?user.id=1369adf90e6146c298833c5db7ff437b&scope.domain.id=default&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 530 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:08.951148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65db92-e180-4069-b593-406596e6093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:08.951809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65db92-e180-4069-b593-406596e6093b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:08.952089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65db92-e180-4069-b593-406596e6093b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:08.952390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65db92-e180-4069-b593-406596e6093b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:08.953592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65db92-e180-4069-b593-406596e6093b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:08.955067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65db92-e180-4069-b593-406596e6093b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:08.962278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 27/56] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:08.968609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:09.075150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a7442-1c44-481c-8eea-a99447986059 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:09.110468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a4b67-acdc-483d-b414-635979c79191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:09.111085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a4b67-acdc-483d-b414-635979c79191 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:09.111367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a4b67-acdc-483d-b414-635979c79191 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:09.111642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a4b67-acdc-483d-b414-635979c79191 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:09.112292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a4b67-acdc-483d-b414-635979c79191 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:09.113750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a4b67-acdc-483d-b414-635979c79191 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:09.207348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 30/57] 10.222.0.15 () {60 vars in 1140 bytes} [Thu Sep 1 17:44:08 2022] POST /identity/v3/projects => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:11.144461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df88f-b412-4c17-9779-89486c4da418 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.145002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df88f-b412-4c17-9779-89486c4da418 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.145002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df88f-b412-4c17-9779-89486c4da418 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.146420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 28/58] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:11.151448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.151805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.152016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.170379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:11.171569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:11.189044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:11.236320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 31/59] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:11.504884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df88f-b412-4c17-9779-89486c4da418 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:11.601815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:11.605508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df88f-b412-4c17-9779-89486c4da418 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:11.637945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa5691-00a3-4304-af2b-f25bb010776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:11.638563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa5691-00a3-4304-af2b-f25bb010776c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.638842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa5691-00a3-4304-af2b-f25bb010776c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.639114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa5691-00a3-4304-af2b-f25bb010776c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.640513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa5691-00a3-4304-af2b-f25bb010776c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:11.641940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa5691-00a3-4304-af2b-f25bb010776c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:11.643482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 29/60] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:11.649533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:11.744729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be7aed-bf9c-4d0a-9f4c-135cd1ce63d4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:11.777382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0ca59-c34a-4afa-9830-a23e71874785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:11.777922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0ca59-c34a-4afa-9830-a23e71874785 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:11.778206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0ca59-c34a-4afa-9830-a23e71874785 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:11.778487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0ca59-c34a-4afa-9830-a23e71874785 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:11.779135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0ca59-c34a-4afa-9830-a23e71874785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:11.780587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0ca59-c34a-4afa-9830-a23e71874785 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:11.880468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 32/61] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:11 2022] POST /identity/v3/users => generated 275 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:14.900904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c951ba1-a268-496d-bd9f-5f36785ba825 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:14.901904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c951ba1-a268-496d-bd9f-5f36785ba825 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:14.902435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c951ba1-a268-496d-bd9f-5f36785ba825 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:14.903994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 30/62] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:14.909299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c2a4-4418-47e1-b279-2ab0542c4dae None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:14.910088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c2a4-4418-47e1-b279-2ab0542c4dae None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:14.910583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c2a4-4418-47e1-b279-2ab0542c4dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:14.941861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-08f1c2a4-4418-47e1-b279-2ab0542c4dae None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:14.943838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-08f1c2a4-4418-47e1-b279-2ab0542c4dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:14.949395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:14.952605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fb153-08be-4a95-8cec-3779205defa7 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:14.952896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fb153-08be-4a95-8cec-3779205defa7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:14.953181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fb153-08be-4a95-8cec-3779205defa7 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:14.954535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 31/63] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:14.959553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162f10-5d18-4c08-8f85-d01d44d3bb58 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:14.960068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162f10-5d18-4c08-8f85-d01d44d3bb58 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:14.960541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162f10-5d18-4c08-8f85-d01d44d3bb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:14.984618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f2162f10-5d18-4c08-8f85-d01d44d3bb58 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:14.985843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f2162f10-5d18-4c08-8f85-d01d44d3bb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:15.048558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 33/64] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:15.100189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 32/65] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:15.342543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ce6fd-15be-4734-a4dc-8b16e08cd77c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.343212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ce6fd-15be-4734-a4dc-8b16e08cd77c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.343683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ce6fd-15be-4734-a4dc-8b16e08cd77c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.345272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 34/66] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:15.360871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ce6fd-15be-4734-a4dc-8b16e08cd77c None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:15.371460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bce04-e1d4-4a37-a39b-6c45e82fdf3c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.372069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bce04-e1d4-4a37-a39b-6c45e82fdf3c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.372560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bce04-e1d4-4a37-a39b-6c45e82fdf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.409158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-788bce04-e1d4-4a37-a39b-6c45e82fdf3c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:15.410796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-788bce04-e1d4-4a37-a39b-6c45e82fdf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:15.490813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 33/67] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:15.496566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3d351-debf-4575-a973-6cb5a2097a9e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.496566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3d351-debf-4575-a973-6cb5a2097a9e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.496566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3d351-debf-4575-a973-6cb5a2097a9e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.496566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 34/68] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:15.496566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3d351-debf-4575-a973-6cb5a2097a9e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:15.517874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ce6fd-15be-4734-a4dc-8b16e08cd77c None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:15.534016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:15.558446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee809589-41f8-4f91-a1e1-a27e76c3f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:15.559407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee809589-41f8-4f91-a1e1-a27e76c3f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.559685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee809589-41f8-4f91-a1e1-a27e76c3f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.560347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee809589-41f8-4f91-a1e1-a27e76c3f8f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.573752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee809589-41f8-4f91-a1e1-a27e76c3f8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:15.579313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ee809589-41f8-4f91-a1e1-a27e76c3f8f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:15.581127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:15.581988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:15.584132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 35/69] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:15.590493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c295b00-dab7-4942-a5dd-79aa9be02ec5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.591272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c295b00-dab7-4942-a5dd-79aa9be02ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.592535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c295b00-dab7-4942-a5dd-79aa9be02ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.614907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3c295b00-dab7-4942-a5dd-79aa9be02ec5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:15.616985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3c295b00-dab7-4942-a5dd-79aa9be02ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:15.632178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3d351-debf-4575-a973-6cb5a2097a9e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:15.678138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:15.679053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.679683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.680107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.694789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:15.700824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:15.701773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 35/70] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:15.707709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:15.709244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 36/71] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:15.709244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c722-2159-47ce-ae64-873b6f216ddf None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.709244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c722-2159-47ce-ae64-873b6f216ddf None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.709244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c722-2159-47ce-ae64-873b6f216ddf None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.709244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 37/72] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:15.712431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023c722-2159-47ce-ae64-873b6f216ddf None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:15.816575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:15.831500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f19b5-d751-44e2-8944-0ada8923738f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:15.835450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023c722-2159-47ce-ae64-873b6f216ddf None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:15.864824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:15.867862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:15.868511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.868792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.869067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.869740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:15.870043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:15.871427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d01bd-1245-42c4-995e-6ec3e98b9576 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:15.871712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:15.872552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.873074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.873482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.874285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:15.874723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:15.876266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:15.883556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 36/73] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:15.884818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:15.885138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:15.885415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:15.890821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 38/74] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:15.891936 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:15.908160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:15.909406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:15.984531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 37/75] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:15.984531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:15.999641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b81dc-adbd-47ad-a9a0-6f7111d8ed81 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.036750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.037384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.037667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.037940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.048614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:16.054070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:16.054914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:16.055702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 39/76] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:16.057004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.082361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.101999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9ddb4-eabe-4785-9818-9c80892c27c3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.124889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.144767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.145716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.146168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.146656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.167099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.168612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.174857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 38/77] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:15 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:16.174857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.203597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad606a-bb52-4de0-8638-2cac1cec5b29 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.240865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.241856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.242146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.242567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.243741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.245219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.251303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 40/78] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:16.251303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.287188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28869c-0cdf-464e-9433-26f4f04c8fe3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.322146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.322820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.323103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.323375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.330336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.334963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:16.339752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:16.342957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 39/79] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:16.343928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.357951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bc3be-68a5-4c54-80a8-cb3cc62a745a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.381058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.393896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.394546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.394838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.395122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.395944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:16.396279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.397725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.410139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 41/80] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:16.410139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.459631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c038c-bf77-4e65-a950-f44960a4ffbd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.501760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.502776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.503051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.503321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.524643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c9512-35c1-48d1-881b-d103a52f5cfa None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.559883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.560571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.560852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.561125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.584595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.587936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.589435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.610825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.630215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.631691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.688961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 40/81] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:16.689870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.730817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 42/82] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:16.731670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.786466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be38e46-5553-401e-b18f-9931d995f0f8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.813510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:16.824354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.824998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.825276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.825545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.826657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.828107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.829870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 41/83] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:16.830668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.831594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5272-2628-4141-b4ba-09f8c273eb18 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.870782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.871315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.871597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.871877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.872583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:16.872890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.874251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.882300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 43/84] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:16.883092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.933724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7eeec-e560-4f17-b2e0-12a1e6dc0fba None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:16.968860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:16.969501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:16.969789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:16.970067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:16.971207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:16.972713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:16.975378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:16.976141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:16.983767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ecc7f-5092-4cc4-affc-b17a23bd62e0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.010681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.017550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.018176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.018457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.018734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.019875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.021972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.023690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 44/86] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:17.024416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.039290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.079452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db8a88-20e0-4159-860d-77afdc72bb7a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.114878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.115516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.115803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.116070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.117247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.118744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.120369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 43/87] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:16 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:17.121200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.122735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1010f-c1fa-40ba-b514-7f55da53a9c3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.159370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.159984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.160271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.160583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.161826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.163667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.221591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b400-8e54-424e-9e99-6fe9779914ce None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.223091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.255217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.255987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.256269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.256570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.257696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.259182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.260799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:17.261562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.271904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 45/89] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:17 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:17.272797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.351196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.383805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331298f-9063-446d-87b2-87bf3ebd0aca None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.389512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6949d83-9089-4bd6-8e56-aa4c0af8295f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.423907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.424583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.424868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.425145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.431780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.432433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.433283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.433283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.434425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=5ad5ace901534fc38bd2b9d0072d016b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:17.434425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.435439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.441340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 46/90] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/role_assignments?user.id=5ad5ace901534fc38bd2b9d0072d016b&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:17.443042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.475746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.515901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.517480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.604707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2aaf5-2229-4799-96ad-90281147ffbb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.650667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.651239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 45/91] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:17.651957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.662170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698987-8697-4435-8e44-015119c477ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.662806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698987-8697-4435-8e44-015119c477ad None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.663094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698987-8697-4435-8e44-015119c477ad None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.663851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698987-8697-4435-8e44-015119c477ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.664696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698987-8697-4435-8e44-015119c477ad None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=1369adf90e6146c298833c5db7ff437b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:17.665436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698987-8697-4435-8e44-015119c477ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.667181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698987-8697-4435-8e44-015119c477ad None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.675398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 47/92] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/role_assignments?user.id=1369adf90e6146c298833c5db7ff437b&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:17.676339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698987-8697-4435-8e44-015119c477ad None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:17.753137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:17.816026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6291c1-a466-4b6d-8938-2ac3c101023d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.885166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.885799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.886080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.886352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.886766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698987-8697-4435-8e44-015119c477ad None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:17.938945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:17.939523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:17.939806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:17.940085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:17.944246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:17.946638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:17.948586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 48/93] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:17.961160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:18.028074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:18.029613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:18.092677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:18.106503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:18.141180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 46/94] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:18.152335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:18.182270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521cc6f-ec1c-499d-be13-87c48d6d14a1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:18.254743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:18.254743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:18.254743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:18.254743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:18.255596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] RBAC: Adding query filter params (role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=5ad5ace901534fc38bd2b9d0072d016b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:18.255596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:18.256448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:18.263270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 49/95] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:17 2022] GET /identity/v3/role_assignments?user.id=5ad5ace901534fc38bd2b9d0072d016b&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:18.313354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285b904-d598-4eb4-9042-645a264b72c0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:18.365458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa013472-ec3e-4d82-9e1b-fbe7e8561bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:18.366119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa013472-ec3e-4d82-9e1b-fbe7e8561bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:18.366398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa013472-ec3e-4d82-9e1b-fbe7e8561bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:18.366675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa013472-ec3e-4d82-9e1b-fbe7e8561bd0 None admin] PATH_INFO: `/v3/projects/e2670650c31b477b92b45bda236ae5c1` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:18.370053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa013472-ec3e-4d82-9e1b-fbe7e8561bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=e2670650c31b477b92b45bda236ae5c1)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:18.371572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa013472-ec3e-4d82-9e1b-fbe7e8561bd0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:18.373354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 47/96] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:18 2022] GET /identity/v3/projects/e2670650c31b477b92b45bda236ae5c1 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:18.379499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:18.512964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2ca2f-cb77-42f3-8404-8b72de003f1a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:18.542054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:18.550297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:18.556863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:18.556863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:18.556863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:18.556863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=e2670650c31b477b92b45bda236ae5c1, user.id=5ad5ace901534fc38bd2b9d0072d016b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:18.556863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:18.556863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f9045-d721-4c3f-8c5d-6e64cdee533e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:18.559822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 50/97] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:18 2022] GET /identity/v3/role_assignments?user.id=5ad5ace901534fc38bd2b9d0072d016b&scope.project.id=e2670650c31b477b92b45bda236ae5c1&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:19.741757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69c8be-8c18-45ed-9437-c613590b66cb None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:19.742775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69c8be-8c18-45ed-9437-c613590b66cb None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:19.743255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69c8be-8c18-45ed-9437-c613590b66cb None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:19.744960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 48/98] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:19.750324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30c076-2965-4900-b1dd-61c346421d24 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:19.751149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30c076-2965-4900-b1dd-61c346421d24 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:19.751617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30c076-2965-4900-b1dd-61c346421d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:19.771070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4b30c076-2965-4900-b1dd-61c346421d24 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:19.772819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4b30c076-2965-4900-b1dd-61c346421d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:19.863735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 51/99] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:20.023705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5ff0-bb71-47a7-9f81-8497f2beaf87 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.024297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5ff0-bb71-47a7-9f81-8497f2beaf87 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.024798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa5ff0-bb71-47a7-9f81-8497f2beaf87 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.026557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 49/100] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:20.031947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852119de-0980-4149-ba70-4f282fd4e2e7 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.032511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852119de-0980-4149-ba70-4f282fd4e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.032999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852119de-0980-4149-ba70-4f282fd4e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.055515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-852119de-0980-4149-ba70-4f282fd4e2e7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:20.057035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-852119de-0980-4149-ba70-4f282fd4e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:20.136115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 52/101] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:20.397768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa5ff0-bb71-47a7-9f81-8497f2beaf87 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:20.449059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:20.508275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f17f1-e220-4630-bdfc-23cb41b4c9be None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.508969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f17f1-e220-4630-bdfc-23cb41b4c9be None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.509501 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f17f1-e220-4630-bdfc-23cb41b4c9be None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.511055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 53/102] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:20.522008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25c869-d5fd-4c7b-a05f-049df4ea9c36 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.522634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25c869-d5fd-4c7b-a05f-049df4ea9c36 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.523098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25c869-d5fd-4c7b-a05f-049df4ea9c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.541057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa5ff0-bb71-47a7-9f81-8497f2beaf87 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:20.565393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1d25c869-d5fd-4c7b-a05f-049df4ea9c36 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:20.566749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1d25c869-d5fd-4c7b-a05f-049df4ea9c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:20.612746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7499452-b72c-44ee-8c53-f95bba31e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:20.614371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7499452-b72c-44ee-8c53-f95bba31e2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.615029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7499452-b72c-44ee-8c53-f95bba31e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.615311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7499452-b72c-44ee-8c53-f95bba31e2d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.637495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7499452-b72c-44ee-8c53-f95bba31e2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-b7499452-b72c-44ee-8c53-f95bba31e2d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:20.643960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:20.644782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:20.647668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 50/103] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:20.663056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80c47-46d3-4d54-bf7a-1d561ef6ed8b None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.663614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80c47-46d3-4d54-bf7a-1d561ef6ed8b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.664078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80c47-46d3-4d54-bf7a-1d561ef6ed8b None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.664670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:20.666483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 51/104] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:20.667374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f80c47-46d3-4d54-bf7a-1d561ef6ed8b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:20.700397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 54/105] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:20.702104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.702394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.702670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.723195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:20.725445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:20.802502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 55/106] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:20.803419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:20.877947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:20.890066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f80c47-46d3-4d54-bf7a-1d561ef6ed8b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:20.971661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:20.972627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:20.973087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:20.973617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:20.974525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:20.975007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:20.977214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:20.995937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 52/107] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:20.999212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf255f-45eb-47b7-9e03-7e95cecb1ec4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.002019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.029752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:21.062470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.063128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.063409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.063684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.088743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:21.095262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:21.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:21.096949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:21.099340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 56/108] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:21.099661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.193053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8659f-7f3e-4423-ae91-08773ff80549 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.229967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.230692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.231070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.231425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.232616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.234058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.236341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 53/109] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:21.241459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.258344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14245e9-1627-4d8d-abcb-bcdeadff32ef None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.274800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:21.313096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.314382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.314858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.315323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.330517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:21.338671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:21.339525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:21.340283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:21.342022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 57/110] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:21.343767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.357155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4c31-4dcf-432c-8e4b-3c552e6b1e49 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.388692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:21.392548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.393216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.393501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.393857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.394444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:21.394749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.396726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.405785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 54/111] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:21.406651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.481252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00bf89-d638-4d8d-9229-bc874b23ef49 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.504629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:21.514904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e1238-3a3d-4a5e-9c7f-8a4a50d4c347 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.518021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.518330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.518615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.518888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.520023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.522031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.524058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 58/112] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:21.524808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.560382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.561468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.562705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.563160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.564518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.566154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.573003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 55/113] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:21.573003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.624931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:21.627753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec0ac-cd9b-4727-afb1-ac8989256414 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.662886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.663490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.663771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.664046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.664743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:21.665046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.666415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc707c6-0084-4061-99fc-535517d2f5b1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.674243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 59/114] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:21.674776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e193f-ba68-483c-946f-c2454b339d8d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.680855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c573-ba5e-4763-b8c2-5df73a9060b4 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.681152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c573-ba5e-4763-b8c2-5df73a9060b4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.681438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c573-ba5e-4763-b8c2-5df73a9060b4 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.682719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 60/115] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:21.683415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c573-ba5e-4763-b8c2-5df73a9060b4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.708764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df27bf-41d1-43e9-832b-03b1e4eef707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.709449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df27bf-41d1-43e9-832b-03b1e4eef707 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.709733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df27bf-41d1-43e9-832b-03b1e4eef707 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.710009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df27bf-41d1-43e9-832b-03b1e4eef707 None admin] PATH_INFO: `/v3/users/204c36d3aa8d4dfaa4b958a81a1652e1` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.724009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df27bf-41d1-43e9-832b-03b1e4eef707 None admin] RBAC: Authorizing `identity:get_user(user_id=204c36d3aa8d4dfaa4b958a81a1652e1)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.725644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df27bf-41d1-43e9-832b-03b1e4eef707 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.727257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 56/116] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/users/204c36d3aa8d4dfaa4b958a81a1652e1 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:21.728419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f6029-8fdc-4212-811f-b44ce0fbd0b1 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.728701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f6029-8fdc-4212-811f-b44ce0fbd0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.728978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f6029-8fdc-4212-811f-b44ce0fbd0b1 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.730307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 57/117] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:21.730846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f6029-8fdc-4212-811f-b44ce0fbd0b1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.749259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:21.787912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c573-ba5e-4763-b8c2-5df73a9060b4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.822012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.822607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.822890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.823162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.823810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.825258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.834311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f6029-8fdc-4212-811f-b44ce0fbd0b1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:21.873390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74bfd1-831c-4bf7-b8a5-c026004777e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:21.874107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74bfd1-831c-4bf7-b8a5-c026004777e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:21.874348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74bfd1-831c-4bf7-b8a5-c026004777e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:21.874629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74bfd1-831c-4bf7-b8a5-c026004777e3 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576/users/5ad5ace901534fc38bd2b9d0072d016b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:21.876586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74bfd1-831c-4bf7-b8a5-c026004777e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7de433353c124372b395806b9f1eb576, user_id=5ad5ace901534fc38bd2b9d0072d016b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:21.880119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74bfd1-831c-4bf7-b8a5-c026004777e3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:21.977599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 61/118] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:44:21 2022] POST /identity/v3/projects => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:21.978440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:21.995503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.010325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 58/119] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:21 2022] PUT /identity/v3/projects/7de433353c124372b395806b9f1eb576/users/5ad5ace901534fc38bd2b9d0072d016b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:22.011512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185584c-e757-472d-9527-933393d4d197 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.011850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185584c-e757-472d-9527-933393d4d197 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.012135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185584c-e757-472d-9527-933393d4d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.038541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2185584c-e757-472d-9527-933393d4d197 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:22.039941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2185584c-e757-472d-9527-933393d4d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:22.127351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf301-631c-4c24-839f-b985e3c33a9c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.149853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 59/120] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:22.152487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185584c-e757-472d-9527-933393d4d197 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.165383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.182803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b820c-6c55-452c-b15d-2ff2a067c6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.183928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b820c-6c55-452c-b15d-2ff2a067c6ef None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.184489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b820c-6c55-452c-b15d-2ff2a067c6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.185001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b820c-6c55-452c-b15d-2ff2a067c6ef None admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.186679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b820c-6c55-452c-b15d-2ff2a067c6ef None admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:22.188509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b820c-6c55-452c-b15d-2ff2a067c6ef None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:22.190168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 62/121] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:21 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:22.191334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.191624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.191901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.213165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:22.214373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:22.282391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185584c-e757-472d-9527-933393d4d197 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.308606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 63/122] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:22.309552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.360781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.361782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.362265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.362740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.363106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.464889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b9e10-0616-47d9-a026-24fb31d1f528 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.483421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:22.493374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:22.504774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.510162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.510851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.511095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.511373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.513719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:22.515237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:22.517303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 64/123] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:22.538678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.608490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 60/124] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:22.609696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.662599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f6220-728f-4372-9699-0396fbbc55f9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.666235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.717115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cab469-0442-42f3-890e-100090333bb9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.737038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.737730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.738090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.738377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576/users/1369adf90e6146c298833c5db7ff437b/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.740660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:22.744211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] RBAC: Authorizing `identity:create_grant(project_id=7de433353c124372b395806b9f1eb576, user_id=1369adf90e6146c298833c5db7ff437b, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:22.745701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:22.756499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.757338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.757732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.758088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.770032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:22.776073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:22.777010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 61/125] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:22.778205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.876422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 65/126] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:22 2022] PUT /identity/v3/projects/7de433353c124372b395806b9f1eb576/users/1369adf90e6146c298833c5db7ff437b/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:22.876422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:22.913169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bb461-95db-41fb-ba19-9d31bd03ed8d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:22.987001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:22.987655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:22.987940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:22.988217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:22.990808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:22.992306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:22.994416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 62/127] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:22.995067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.064725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:23.076842 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14d24a-0e07-4219-992a-5a8197418158 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.127165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.128017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.128330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.128618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.139328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:23.143505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:23.152927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:23.157189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:23.157995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 66/128] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:23.159271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97398c-86fa-4b97-8802-6e70bd56ef98 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.159346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.200110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.201087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.201561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.202048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.202907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:23.203399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.205101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.215017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 63/129] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:23.221201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.293193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2568035-6012-4434-a162-cf267c6a8140 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.332883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.334130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.334690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.335167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.336105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] RBAC: Adding query filter params (role.id=c61c993396164231915b06811bea6931, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=204c36d3aa8d4dfaa4b958a81a1652e1) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:23.336656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.338328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.344407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 67/130] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:23 2022] GET /identity/v3/role_assignments?user.id=204c36d3aa8d4dfaa4b958a81a1652e1&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=c61c993396164231915b06811bea6931 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:23.345526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.380262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:23.400078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd384bc-d585-4c0e-8fa2-b084df261b36 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.440869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:23.483571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ec22-d793-4b0b-9369-8d9b9adbf62a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.490970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.491622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.491913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.492188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.493095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:23.493412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.494846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.518610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 64/131] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:23.524714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.537953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.538624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.538909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.539188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.540377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.541847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.543463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 68/132] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:23 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:23.550613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.692531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f323df-64c4-41af-9b7b-04b1a203a73a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.732950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.733918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.734215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.734499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.735811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.737453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.739291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 65/133] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:23 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:23.744851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.746288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935836f-0f7e-4da3-a2b6-bb34eea7341f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.778044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:23.801304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:23.811255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.812047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.812488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.812967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] PATH_INFO: `/v3/projects/e2670650c31b477b92b45bda236ae5c1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.816361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] RBAC: Authorizing `identity:get_project(project_id=e2670650c31b477b92b45bda236ae5c1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.818004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.819897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 69/134] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:23 2022] GET /identity/v3/projects/e2670650c31b477b92b45bda236ae5c1 => generated 312 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:23.826348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:23.866588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa567e-b8c3-4d5f-9ec0-b9b088d0338e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:23.913604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:23.914357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:23.914549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:23.914800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:23.916863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:23.919143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:23.922564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 66/135] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:23 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:23.927406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:24.020096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6389ed7-084f-42d8-9d41-2caf65780f56 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:24.085130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1064c9-93e3-4d60-b186-35860e7e7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:24.085798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1064c9-93e3-4d60-b186-35860e7e7015 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:24.086079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1064c9-93e3-4d60-b186-35860e7e7015 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:24.086356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1064c9-93e3-4d60-b186-35860e7e7015 None admin] PATH_INFO: `/v3/projects/e2670650c31b477b92b45bda236ae5c1/users/5ad5ace901534fc38bd2b9d0072d016b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:24.089440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1064c9-93e3-4d60-b186-35860e7e7015 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2670650c31b477b92b45bda236ae5c1, user_id=5ad5ace901534fc38bd2b9d0072d016b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:24.091125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1064c9-93e3-4d60-b186-35860e7e7015 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:24.099059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:24.106292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2e71-7b9b-4507-b4a4-7ae87a32ec29 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:24.146721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb16847-26c1-4122-a284-2e539d126e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:24.147392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb16847-26c1-4122-a284-2e539d126e21 None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:24.147674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb16847-26c1-4122-a284-2e539d126e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:24.147951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb16847-26c1-4122-a284-2e539d126e21 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576/users/5ad5ace901534fc38bd2b9d0072d016b/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:24.150337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb16847-26c1-4122-a284-2e539d126e21 None admin] RBAC: Authorizing `identity:create_grant(project_id=7de433353c124372b395806b9f1eb576, user_id=5ad5ace901534fc38bd2b9d0072d016b, role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:24.157199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb16847-26c1-4122-a284-2e539d126e21 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:24.228153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 70/136] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:23 2022] PUT /identity/v3/projects/e2670650c31b477b92b45bda236ae5c1/users/5ad5ace901534fc38bd2b9d0072d016b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:24.259799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 67/137] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:23 2022] PUT /identity/v3/projects/7de433353c124372b395806b9f1eb576/users/5ad5ace901534fc38bd2b9d0072d016b/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:24.423536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c41ee8-ef66-45b8-80d7-e7ef319b4ef3 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:24.423932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c41ee8-ef66-45b8-80d7-e7ef319b4ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:24.424163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c41ee8-ef66-45b8-80d7-e7ef319b4ef3 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:24.425680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 71/138] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:24.432231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b218f3-b26d-448d-994b-2056cc9b0cdc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:24.432544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b218f3-b26d-448d-994b-2056cc9b0cdc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:24.432827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b218f3-b26d-448d-994b-2056cc9b0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:24.458457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a1b218f3-b26d-448d-994b-2056cc9b0cdc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:24.459941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a1b218f3-b26d-448d-994b-2056cc9b0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:24.561054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 68/139] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:24.755250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0068-5c08-4ae1-baf0-5e9899b619de None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:24.755484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0068-5c08-4ae1-baf0-5e9899b619de None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:24.755822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c0068-5c08-4ae1-baf0-5e9899b619de None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:24.757353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 72/140] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:24.762839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:24.763139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:24.763427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:24.789329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:24.790545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:24.873156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0068-5c08-4ae1-baf0-5e9899b619de None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:24.893983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 69/141] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:24.978581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:25.034311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c0068-5c08-4ae1-baf0-5e9899b619de None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:25.090899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:25.091564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.091848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.092122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.104564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:25.110115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:25.111935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:25.113037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 73/142] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:25.121631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:25.139083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:25.250226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146f2b7-d014-4384-9b7c-f6cb1c17c7d8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:25.290122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:25.290122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.290122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.290122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.290587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:25.291027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:25.292521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:25.301978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 70/143] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:25.308335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:25.415116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542229-101c-40f7-b08e-1029bd3537d5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:25.436198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:25.450573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:25.451206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.462013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.462013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.462013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:25.462013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:25.462013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 74/144] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:25 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:25.462568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:25.499588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:25.613161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:25.659089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d82c3-dd31-4d9c-9052-b2c2d84c0dbb None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:25.664167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097d621-61e6-44fe-8ba7-564f90552462 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:25.703245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:25.703950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.704244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.704718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.706060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:25.707860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:25.710228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 75/145] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:25.715901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:25.744257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c455ae-f1bb-48ee-a3f4-e45fd383904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:25.745365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c455ae-f1bb-48ee-a3f4-e45fd383904f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.745857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c455ae-f1bb-48ee-a3f4-e45fd383904f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.746333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c455ae-f1bb-48ee-a3f4-e45fd383904f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.778484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:25.825174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7bb91-9f2f-4a34-bdaa-b345f1f253e1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:25.858337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c455ae-f1bb-48ee-a3f4-e45fd383904f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:25.860049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c455ae-f1bb-48ee-a3f4-e45fd383904f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:25.863701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce830b-ab3e-4ad3-95a3-1ba951f02aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:25.863701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce830b-ab3e-4ad3-95a3-1ba951f02aaf None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.864171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce830b-ab3e-4ad3-95a3-1ba951f02aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.864842 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce830b-ab3e-4ad3-95a3-1ba951f02aaf None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.865660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce830b-ab3e-4ad3-95a3-1ba951f02aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:25.867235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce830b-ab3e-4ad3-95a3-1ba951f02aaf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:25.946188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:25.975103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 71/146] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:25.976799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd723c-f051-4c00-a96a-aa8cb7b456e9 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:25.977225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd723c-f051-4c00-a96a-aa8cb7b456e9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:25.977663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd723c-f051-4c00-a96a-aa8cb7b456e9 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:25.979088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 72/147] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:25.983777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd723c-f051-4c00-a96a-aa8cb7b456e9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:25.998170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 76/148] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:25 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:25.999559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890229-5f16-462d-9377-442465f346e9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.004876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890229-5f16-462d-9377-442465f346e9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.005480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890229-5f16-462d-9377-442465f346e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.039870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-cd890229-5f16-462d-9377-442465f346e9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:26.040663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cd890229-5f16-462d-9377-442465f346e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:26.120598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 77/149] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:26.157967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd723c-f051-4c00-a96a-aa8cb7b456e9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:26.211139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:26.212050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.212503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.212920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.216977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:26.218582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:26.220396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 73/150] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:25 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:26.229410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890229-5f16-462d-9377-442465f346e9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:26.299153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:26.361199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890229-5f16-462d-9377-442465f346e9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:26.399699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:26.405070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.405070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.405070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.405070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=5ad5ace901534fc38bd2b9d0072d016b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:26.405070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:26.405070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064dfe99-afb5-42a6-8e8c-237007397495 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:26.409940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 78/151] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:26 2022] GET /identity/v3/role_assignments?user.id=5ad5ace901534fc38bd2b9d0072d016b&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:26.585846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:26.616097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678131f-3abd-40f4-8670-697ff21b81be None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.616858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678131f-3abd-40f4-8670-697ff21b81be None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.617340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678131f-3abd-40f4-8670-697ff21b81be None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.619476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 79/152] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:26.629178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.629822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.630441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.646142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:26.665459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:26.666739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:26.736489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fe32f-721e-4fd7-8bde-f94242b5739a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:26.791083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 80/153] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:26.793137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:26.793794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.794077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.794351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.807277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:26.812347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:26.813484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:26.814515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:26.816738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 74/154] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:26.822906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:26.866596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:26.946889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1e0ed-3e7e-4c3c-a9a0-df429a21acc8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:26.987207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:26.987829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:26.988107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:26.988404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:26.989164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:26.989468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:26.990834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50167a06-31ca-4cbb-9775-6173fe6479e6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:26.999799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 81/155] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:27.006331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.122972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2711f-d5dd-45e3-9c03-0b9c06d70581 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.123484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2711f-d5dd-45e3-9c03-0b9c06d70581 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.123574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2711f-d5dd-45e3-9c03-0b9c06d70581 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.125053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 82/156] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:27.132150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.132487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.132767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.145947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc6beb-a392-4916-b500-bac32d794b1b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:27.149792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:27.156960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:27.158178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:27.186966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c6ec4-3b55-4088-bde1-f0ae4f90b657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:27.187619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c6ec4-3b55-4088-bde1-f0ae4f90b657 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.187901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c6ec4-3b55-4088-bde1-f0ae4f90b657 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.188175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c6ec4-3b55-4088-bde1-f0ae4f90b657 None admin] PATH_INFO: `/v3/users/1369adf90e6146c298833c5db7ff437b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.189511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c6ec4-3b55-4088-bde1-f0ae4f90b657 None admin] RBAC: Authorizing `identity:get_user(user_id=1369adf90e6146c298833c5db7ff437b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:27.191171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c6ec4-3b55-4088-bde1-f0ae4f90b657 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:27.193862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 75/157] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/users/1369adf90e6146c298833c5db7ff437b => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:27.198326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257641-8790-429d-a1cb-a9d3c15b71b5 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.198326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257641-8790-429d-a1cb-a9d3c15b71b5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.198326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257641-8790-429d-a1cb-a9d3c15b71b5 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.198326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 76/158] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:27.198326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d257641-8790-429d-a1cb-a9d3c15b71b5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.262363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 83/159] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:27.263331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.306430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:27.341813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d257641-8790-429d-a1cb-a9d3c15b71b5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:27.392480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf8603-f652-42d8-aa1e-511bbc659436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:27.393175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8603-f652-42d8-aa1e-511bbc659436 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.393462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8603-f652-42d8-aa1e-511bbc659436 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.393741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8603-f652-42d8-aa1e-511bbc659436 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.407236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf8603-f652-42d8-aa1e-511bbc659436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:27.408757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b0f97-81de-479d-94df-ac34547a975a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-0ccf8603-f652-42d8-aa1e-511bbc659436 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:27.413787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:27.414682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:27.415535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 77/160] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:27.420999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.421450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.421739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.446781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:27.448036 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:27.458118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:27.458994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.459460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.461264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.484411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:27.548826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:27.551116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:27.582198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 78/161] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:27.583664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.590441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:27.684385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 84/162] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:27.685289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.817449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86fb2-6525-489e-b59c-8c83433d458b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:27.888974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:27.889592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.889874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.891763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.892472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:27.892783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:27.904450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:27.906442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0339db3-395c-4ab6-b385-e57c5401a937 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:27.916511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 79/163] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:27.917213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:27.960723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:27.966470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:27.976824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:27.979163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:27.979163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:27.979163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:27.979583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:27.980695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:27.982503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 85/164] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:27.983133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.051509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735ffc-02df-475c-9ddc-7276d44b43c8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.089493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.090314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.090816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.091284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.110595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:28.118293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:28.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:28.120044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 80/165] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:28.122029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.207250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bed36e-48ee-4fdf-8fe9-8dff6cd56d4b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.223238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:28.247630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580d08-146a-4d03-9590-a268d20b9d4e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.278033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.278684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.278968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.279248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] PATH_INFO: `/v3/users/204c36d3aa8d4dfaa4b958a81a1652e1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.280419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] RBAC: Authorizing `identity:get_user(user_id=204c36d3aa8d4dfaa4b958a81a1652e1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.281863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.283456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 86/166] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:27 2022] GET /identity/v3/users/204c36d3aa8d4dfaa4b958a81a1652e1 => generated 289 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:28.284224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.299501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.300495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.300996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.302045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.331759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:28.348456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:28.349454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 81/167] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:28.350305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.440759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:28.448843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a1836-4bf7-4114-879e-a1513fadb244 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.514397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.515016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.515319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.515836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.516605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=1369adf90e6146c298833c5db7ff437b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:28.516915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.518340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.526241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 87/168] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:28 2022] GET /identity/v3/role_assignments?user.id=1369adf90e6146c298833c5db7ff437b&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:28.527053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.560170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9205e-e922-4309-a612-3d6691744a53 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.638749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.639375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.639656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.639931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.640910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:28.641212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.642636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.666581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 82/169] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:28.667527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.687979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:28.758547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf22ae-e236-4417-a6f5-51d823322634 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.805008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:28.813971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843bd89-6b09-4993-818d-aa367e1975b5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:28.819009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.819660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.819939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.820214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.822346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.823901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.825709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 88/170] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:28 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:28.827777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:28.859537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:28.860176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:28.860484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:28.860758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:28.861445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:28.861752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:28.863176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:28.887297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 83/171] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:28.889007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.026975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d65b12-c86f-4232-b2ea-8e19a1dd4a95 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:29.032668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:29.081797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c53c-c77d-4b3a-89da-75c1727b8e40 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:29.105943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:29.106595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.106876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.107154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.113007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:29.114502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:29.116147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 89/172] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:28 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:29.116960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.130442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:29.131177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.131464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.131742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576/users/204c36d3aa8d4dfaa4b958a81a1652e1/roles/c61c993396164231915b06811bea6931` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.133869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] RBAC: Authorizing `identity:create_grant(project_id=7de433353c124372b395806b9f1eb576, user_id=204c36d3aa8d4dfaa4b958a81a1652e1, role_id=c61c993396164231915b06811bea6931)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:29.135402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:29.202719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:29.218094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 84/173] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:28 2022] PUT /identity/v3/projects/7de433353c124372b395806b9f1eb576/users/204c36d3aa8d4dfaa4b958a81a1652e1/roles/c61c993396164231915b06811bea6931 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:29.218987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.274390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435ff56-007c-4d42-a157-c647ad3caf47 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:29.355746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb63d9-983b-4450-b630-9035dc78f364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:29.356481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb63d9-983b-4450-b630-9035dc78f364 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.356771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb63d9-983b-4450-b630-9035dc78f364 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.357253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb63d9-983b-4450-b630-9035dc78f364 None admin] PATH_INFO: `/v3/users/5ad5ace901534fc38bd2b9d0072d016b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.361095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb63d9-983b-4450-b630-9035dc78f364 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad5ace901534fc38bd2b9d0072d016b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:29.362555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb63d9-983b-4450-b630-9035dc78f364 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:29.363981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153bc86-0b14-4a3e-964f-23f79ac57082 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:29.364468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 90/174] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:29 2022] GET /identity/v3/users/5ad5ace901534fc38bd2b9d0072d016b => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:29.365812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb1828-0044-4fcb-83f7-10167ebce8ca None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.366097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb1828-0044-4fcb-83f7-10167ebce8ca None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.366424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb1828-0044-4fcb-83f7-10167ebce8ca None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.371273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 91/175] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:29.377490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb1828-0044-4fcb-83f7-10167ebce8ca None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.406270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:29.417732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04db6-c968-4800-8f08-7bd8a40b899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:29.418760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04db6-c968-4800-8f08-7bd8a40b899d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.419226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04db6-c968-4800-8f08-7bd8a40b899d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.426690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04db6-c968-4800-8f08-7bd8a40b899d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.510380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04db6-c968-4800-8f08-7bd8a40b899d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:29.512924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04db6-c968-4800-8f08-7bd8a40b899d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:29.568548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb1828-0044-4fcb-83f7-10167ebce8ca None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:29.619026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:29.628334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:29.628334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.628334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.628334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.691019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 85/176] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:29.696877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:29.697450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:29.697919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:29.706712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:29.706712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:29.737390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:29.738612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:29.804007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:29.821478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 92/177] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:29.822107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.861417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:29.897833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 86/178] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:29.898703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:29.978210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7bd70-a718-4907-b68a-c8cddfead8c4 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.022556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bc220-546a-4bf8-906e-6d251dc0ce2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.023530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bc220-546a-4bf8-906e-6d251dc0ce2a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.024041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bc220-546a-4bf8-906e-6d251dc0ce2a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.024532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bc220-546a-4bf8-906e-6d251dc0ce2a None admin] PATH_INFO: `/v3/projects/e2670650c31b477b92b45bda236ae5c1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.025443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826edc0b-893b-451e-9711-99b97af44ca7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.026084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bc220-546a-4bf8-906e-6d251dc0ce2a None admin] RBAC: Authorizing `identity:get_project(project_id=e2670650c31b477b92b45bda236ae5c1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.027696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bc220-546a-4bf8-906e-6d251dc0ce2a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.029597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 93/179] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:29 2022] GET /identity/v3/projects/e2670650c31b477b92b45bda236ae5c1 => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.031089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624a23-d795-4363-8a24-52b89a0ca245 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.031583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624a23-d795-4363-8a24-52b89a0ca245 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.032045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624a23-d795-4363-8a24-52b89a0ca245 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.033471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 94/180] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:30.034938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b786f-161c-419a-bd0c-68c1b66df8cb None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.035417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b786f-161c-419a-bd0c-68c1b66df8cb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.036466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b786f-161c-419a-bd0c-68c1b66df8cb None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.037878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 95/181] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:30.038848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b786f-161c-419a-bd0c-68c1b66df8cb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.083554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdcae9-cde6-4ade-a879-c6d087f6a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.084095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdcae9-cde6-4ade-a879-c6d087f6a020 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.084419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdcae9-cde6-4ade-a879-c6d087f6a020 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.084699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdcae9-cde6-4ade-a879-c6d087f6a020 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.085843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdcae9-cde6-4ade-a879-c6d087f6a020 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.087286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdcae9-cde6-4ade-a879-c6d087f6a020 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.089501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 87/182] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:29 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.091096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.091380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.091653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.116276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:30.117751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:30.145865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.152719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.156731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b786f-161c-419a-bd0c-68c1b66df8cb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.193694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.194640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.195083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.195510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.196435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=e2670650c31b477b92b45bda236ae5c1, user.id=5ad5ace901534fc38bd2b9d0072d016b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:30.196914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.198588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46f869-467d-42ee-a4c8-13284c874599 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.204717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 96/183] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/role_assignments?user.id=5ad5ace901534fc38bd2b9d0072d016b&scope.project.id=e2670650c31b477b92b45bda236ae5c1&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.205745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 88/184] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:30.206618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.206970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.207261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.207623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.245474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:30.246692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:30.374796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 97/185] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:30.376374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.403114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9535a9-4849-46c5-be38-e3f77c724cc1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.450079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.450830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.451111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.451384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.452123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] RBAC: Adding query filter params (role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=5ad5ace901534fc38bd2b9d0072d016b) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:30.452457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.453918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.471111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 89/186] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/role_assignments?user.id=5ad5ace901534fc38bd2b9d0072d016b&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.549068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.556818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.563799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6b163-21fc-4678-9092-a46df19da5bf None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.596885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.633381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.634101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.634375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.634638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.663513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:30.673144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:30.674321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:30.675210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 98/187] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:30.681386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.757959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b8030-8c79-4652-a581-dd09f54a645c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.799379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.799692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.799974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.800240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.812811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:30.813083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091dd4e-b279-4550-bf2a-8439bf9a6742 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:30.821925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:30.822827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:30.823686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:30.825693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 90/188] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:30.826715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.855485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.856673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.856967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.857241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.858068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:30.858674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.860341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.875818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 99/189] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.875818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.900359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.909200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:30.929432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1218624-ecf6-4477-87d8-fab114650063 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:30.968101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:30.968758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:30.969035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:30.969305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:30.970410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:30.971836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:30.977995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 91/190] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:30.977995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:30.997684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412aae-16d4-4cf0-86ee-e51e6c1cf10e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.042161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.042782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.043178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.043455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.044244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:31.044576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.045930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.059389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 100/191] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:31.059389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.092008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82f0b7-00bf-4f1c-9e30-c9ddb4b4a743 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.126447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.127078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.127359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.127634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] PATH_INFO: `/v3/users/0cc71147272a472ba402d53f4b4f0503` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.141610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.143081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] RBAC: Authorizing `identity:get_user(user_id=0cc71147272a472ba402d53f4b4f0503)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.144583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.146486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 92/192] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:30 2022] GET /identity/v3/users/0cc71147272a472ba402d53f4b4f0503 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:31.147279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.157544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.168717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b076121-0495-4c19-b4f9-b325ebad5f02 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.205129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.206023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.206447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.206877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.207541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.209024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.253107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883f831-b26f-41f5-91d4-05b8ad1a4512 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.289709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.290429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.290706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.290977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] PATH_INFO: `/v3/users/0cc71147272a472ba402d53f4b4f0503` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.292089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] RBAC: Authorizing `identity:get_user(user_id=0cc71147272a472ba402d53f4b4f0503)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.293614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.295214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 93/193] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:31 2022] GET /identity/v3/users/0cc71147272a472ba402d53f4b4f0503 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:31.296013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.328553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 101/194] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:31 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:31.333416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.393500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.425578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f09c-7682-4971-a318-542059ff511d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.464098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93670-85f9-48b5-a18b-d76bc37e9ed2 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.466070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb85d37-c608-46e3-a37c-caf578f84616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.466906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb85d37-c608-46e3-a37c-caf578f84616 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.467257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb85d37-c608-46e3-a37c-caf578f84616 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.468538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb85d37-c608-46e3-a37c-caf578f84616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.503582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8abfea-6595-4860-883c-eb3bd09ccfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.504513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abfea-6595-4860-883c-eb3bd09ccfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.504939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abfea-6595-4860-883c-eb3bd09ccfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.505357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abfea-6595-4860-883c-eb3bd09ccfe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.508327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.549839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb85d37-c608-46e3-a37c-caf578f84616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.552109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb85d37-c608-46e3-a37c-caf578f84616 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.579987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8abfea-6595-4860-883c-eb3bd09ccfe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.581873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8abfea-6595-4860-883c-eb3bd09ccfe6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.639715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.671577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 94/195] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:31.673566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81408d34-72b6-4298-b984-3120ab440a76 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.674000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81408d34-72b6-4298-b984-3120ab440a76 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.674290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81408d34-72b6-4298-b984-3120ab440a76 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.680256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 95/196] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:31.681633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81408d34-72b6-4298-b984-3120ab440a76 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.715798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 102/197] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:31.717896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.718123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.718520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.750363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:31.751562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:31.790235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:31.813096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81408d34-72b6-4298-b984-3120ab440a76 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.848832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 103/198] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:31.848832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.870068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:31.871040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:31.871526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:31.872059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:31.873850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:31.875626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:31.878015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 96/199] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:31 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:31.889689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:31.962866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31249565-b6f5-4ca3-b7aa-825a5ff83419 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:31.996688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:32.001535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.002404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.002604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.002959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.004091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.005942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.007949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 104/200] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:31 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:32.013486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.079350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:32.119765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d596b8c-680e-416e-a28d-7544df3e9943 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.191791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96447577-db23-4648-8fae-80a4a1422bcf None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.213104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.213647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.213923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.214192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.214927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=0cc71147272a472ba402d53f4b4f0503) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:32.215229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.216687 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.231342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 97/201] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:31 2022] GET /identity/v3/role_assignments?user.id=0cc71147272a472ba402d53f4b4f0503&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:32.233902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.237097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.237097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.237097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.237097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.237638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] RBAC: Adding query filter params (role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=0cc71147272a472ba402d53f4b4f0503) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:32.237887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.247217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.258631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 105/202] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/role_assignments?user.id=0cc71147272a472ba402d53f4b4f0503&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:32.449287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef852d1c-1dc5-4577-a7fc-5a30bd8cdb42 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.456553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:32.485463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.486031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.486316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.486593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.499920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:32.506731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:32.507635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:32.508564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 98/203] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:32.514012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.647207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:32.653315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a819bb6-9668-4fd3-a57b-76288456d0f1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.698023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.698645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.698918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.699184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.699860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:32.700157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.701531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.718636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 106/204] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:32.727875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:32.885995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9863c2e-0a28-4057-ba14-e0ed41ef02a5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:32.937151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:32.937791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:32.938069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:32.938340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] PATH_INFO: `/v3/users/204c36d3aa8d4dfaa4b958a81a1652e1` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:32.941548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] RBAC: Authorizing `identity:get_user(user_id=204c36d3aa8d4dfaa4b958a81a1652e1)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:32.943635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:32.945462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 99/205] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/users/204c36d3aa8d4dfaa4b958a81a1652e1 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:32.951110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:33.108303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c9b1-9881-4d92-a65d-b3d0bfa3110b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:33.162695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:33.163242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:33.163521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:33.163795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:33.197910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:33.256045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:33.257501 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:33.408399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 107/206] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:33.414758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:33.488757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:33.594262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5dc81-70d2-47f2-9d99-104f5b599677 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:33.647107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1560bb-ac3f-4eb9-ac4c-9167424b2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:33.647749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1560bb-ac3f-4eb9-ac4c-9167424b2665 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:33.648032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1560bb-ac3f-4eb9-ac4c-9167424b2665 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:33.648334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1560bb-ac3f-4eb9-ac4c-9167424b2665 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:33.649492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1560bb-ac3f-4eb9-ac4c-9167424b2665 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:33.650944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1560bb-ac3f-4eb9-ac4c-9167424b2665 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:33.652936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 100/207] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:33 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:33.658748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:33.718759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559365eb-d7e4-4211-81c6-142e29730506 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:33.719348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559365eb-d7e4-4211-81c6-142e29730506 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:33.719839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559365eb-d7e4-4211-81c6-142e29730506 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:33.721415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 101/208] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:33.726741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ad2e-0c8f-4c95-ab24-873d0263cf56 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:33.727263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ad2e-0c8f-4c95-ab24-873d0263cf56 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:33.727757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ad2e-0c8f-4c95-ab24-873d0263cf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:33.746691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-77b0ad2e-0c8f-4c95-ab24-873d0263cf56 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:33.748190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-77b0ad2e-0c8f-4c95-ab24-873d0263cf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:33.756074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:33.831106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 102/209] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:33.843428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eeb87-c7c0-4e06-9a85-4e157e923733 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:33.883659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:33.884273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:33.884574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:33.884848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:33.885583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] RBAC: Adding query filter params (role.id=c61c993396164231915b06811bea6931, scope.project.id=7de433353c124372b395806b9f1eb576, user.id=204c36d3aa8d4dfaa4b958a81a1652e1) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:33.885892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:33.887309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca91dd-31ae-4d1f-b754-44f421d1e9d1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:33.893622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 108/210] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:33 2022] GET /identity/v3/role_assignments?user.id=204c36d3aa8d4dfaa4b958a81a1652e1&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=c61c993396164231915b06811bea6931 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:34.163669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec977-98ba-4c1b-b923-f3c5a45df868 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.164030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec977-98ba-4c1b-b923-f3c5a45df868 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.164226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec977-98ba-4c1b-b923-f3c5a45df868 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.165795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 103/211] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:34.171132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fd15-aed8-49b2-ae77-ec7cdac531c8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.171419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fd15-aed8-49b2-ae77-ec7cdac531c8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.171690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fd15-aed8-49b2-ae77-ec7cdac531c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.194644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1eb4fd15-aed8-49b2-ae77-ec7cdac531c8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:34.195832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1eb4fd15-aed8-49b2-ae77-ec7cdac531c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:34.230874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec977-98ba-4c1b-b923-f3c5a45df868 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.272147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 109/212] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:34.288676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.411554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec977-98ba-4c1b-b923-f3c5a45df868 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.452009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571016a-9369-4303-b60e-845f3787d397 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.452248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571016a-9369-4303-b60e-845f3787d397 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.452531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571016a-9369-4303-b60e-845f3787d397 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.453885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 110/213] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:34.458646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.459285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.459560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.459831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.461467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.461743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.462008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.476151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:34.481580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:34.481842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:34.482715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:34.483527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:34.484171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:34.486391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 104/214] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:34.493746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.531629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.553037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 111/215] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:34.583244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.593716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.604978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb25803-917c-4d5c-ada6-4cb5b819ca5e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.642704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.643418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.643702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.643977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.644682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:34.645034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.646617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dc3d2-2fb1-4a39-a154-0e0ba5ff125b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:34.655155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 105/216] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:34.656396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057930-6373-4d30-8067-ecb84599a2aa None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.656655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057930-6373-4d30-8067-ecb84599a2aa None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.656921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057930-6373-4d30-8067-ecb84599a2aa None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.658302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 106/217] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:34.665245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057930-6373-4d30-8067-ecb84599a2aa None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.686087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ec5d4-2c90-405e-9229-9307d69803b9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.725120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e8ad-6af9-428e-b3b2-fa216079dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.725773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e8ad-6af9-428e-b3b2-fa216079dca7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.726049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e8ad-6af9-428e-b3b2-fa216079dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.726456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e8ad-6af9-428e-b3b2-fa216079dca7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.727516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e8ad-6af9-428e-b3b2-fa216079dca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.729344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e8ad-6af9-428e-b3b2-fa216079dca7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:34.730732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 112/218] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:34.731610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.731886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.732153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.750932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:34.752093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:34.787207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057930-6373-4d30-8067-ecb84599a2aa None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.801550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.823307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 113/219] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:34.824236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.829265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.829916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.830197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.830473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] PATH_INFO: `/v3/users/97937bcb36684704a0e1c9f7952ab7d3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.850873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=97937bcb36684704a0e1c9f7952ab7d3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.852371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:34.858522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 107/220] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:34 2022] GET /identity/v3/users/97937bcb36684704a0e1c9f7952ab7d3 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:34.858522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:34.869382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:34.934428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb0240-0de0-4edc-87e6-ad672999cd43 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.962937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407d3e-cf82-489d-affa-0ae836e73ae9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:34.968173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.968811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.969088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.969357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:34.970006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:34.971420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:34.998605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:34.999167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:34.999440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:34.999709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.009772 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:35.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:35.016004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 108/221] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:35.025141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.084966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 114/222] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:34 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:35.085764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.089517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.126462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7e423-5305-4243-9841-f7a3c81b50df None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.141534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.168773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.169532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.169814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.170089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.212956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd397c3-fdbf-45f6-84a4-6eb92ae8d5ce None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.254941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.256114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.256114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.256114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.256495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.256806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:35.262587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.262695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.262786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.268046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 115/223] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.268918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.350364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.388656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 109/224] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:35.389042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.389352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.397024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb3478-a75a-4fe6-8179-2658324bcc86 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.439365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.440291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.440612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.440887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.455613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:35.463794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:35.464764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:35.465562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 116/225] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:35.467143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.500753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f722c-0b05-4602-96b3-449a08ede89f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.537867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.538418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.538694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.538966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] PATH_INFO: `/v3/users/0cc71147272a472ba402d53f4b4f0503` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.540199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] RBAC: Authorizing `identity:get_user(user_id=0cc71147272a472ba402d53f4b4f0503)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.541786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.543745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 110/226] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/users/0cc71147272a472ba402d53f4b4f0503 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.544663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.595932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa92e-5d35-4e49-abf3-bb99bf134b21 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.620773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.637371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.638072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.638401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.639367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.640063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.641882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.643610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 117/227] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.644478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.660160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268359e-8cca-4bf8-9801-9ee967eabc42 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.674126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.705830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.706467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.706995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.707271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.708522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:35.708831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.710638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.722778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 111/228] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.724219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.755113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762889ca-fcc9-4477-9d87-99267761d3ed None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.792418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.793151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.793508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.793852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.795052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.796585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.798472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 118/229] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.804243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.852375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae126e-8e66-4b24-8cbf-d1ce30deb477 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:35.853899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.894643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:35.896108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:35.896564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:35.897105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:35.897980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=97937bcb36684704a0e1c9f7952ab7d3) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:35.898439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:35.900149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:35.906409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 112/230] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/role_assignments?user.id=97937bcb36684704a0e1c9f7952ab7d3&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:35.907416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:35.922082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:35.935926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b52b-77a9-49e8-b87c-b500b043e24a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.015282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.016518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.017055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.017511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] PATH_INFO: `/v3/users/0cc71147272a472ba402d53f4b4f0503` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.018990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=0cc71147272a472ba402d53f4b4f0503)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.021495 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.023617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 119/231] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:35 2022] GET /identity/v3/users/0cc71147272a472ba402d53f4b4f0503 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:36.033357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.087172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2e372-223a-4e97-bcb8-2e122c2fd02c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.144361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.144361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.144361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.144361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/0cc71147272a472ba402d53f4b4f0503/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.144824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc7140fe2ff44d7a9a15836c34f27dfc, user_id=0cc71147272a472ba402d53f4b4f0503, role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.145959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.191603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea905c5-376c-4555-a656-660e15576ce4 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.228981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 113/232] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:35 2022] PUT /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/0cc71147272a472ba402d53f4b4f0503/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:36.229870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011914d9-82af-4535-b288-2b5a8a205571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.230815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011914d9-82af-4535-b288-2b5a8a205571 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.231237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011914d9-82af-4535-b288-2b5a8a205571 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.231647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011914d9-82af-4535-b288-2b5a8a205571 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.233158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011914d9-82af-4535-b288-2b5a8a205571 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.238930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011914d9-82af-4535-b288-2b5a8a205571 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.238930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 120/233] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:36 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:36.247057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:36.300539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:36.405864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c44fd-1d90-44b6-b115-e40f8819de73 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:36.445226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058f475-4cbc-4c44-bab7-6616e87e1715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:36.446226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058f475-4cbc-4c44-bab7-6616e87e1715 None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:36.446697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058f475-4cbc-4c44-bab7-6616e87e1715 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:36.447180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058f475-4cbc-4c44-bab7-6616e87e1715 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/0cc71147272a472ba402d53f4b4f0503/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:36.449351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058f475-4cbc-4c44-bab7-6616e87e1715 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc7140fe2ff44d7a9a15836c34f27dfc, user_id=0cc71147272a472ba402d53f4b4f0503, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:36.451047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058f475-4cbc-4c44-bab7-6616e87e1715 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:36.562044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 114/234] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:36 2022] PUT /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/0cc71147272a472ba402d53f4b4f0503/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:37.404154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b4161-daa2-4f22-861e-e8c671522d33 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.405180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b4161-daa2-4f22-861e-e8c671522d33 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.405662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b4161-daa2-4f22-861e-e8c671522d33 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.407451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 121/235] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:37.412291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.412837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.413303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.433639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908b299-5e73-4f4d-9e13-712d05592cc5 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.434243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908b299-5e73-4f4d-9e13-712d05592cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.434708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908b299-5e73-4f4d-9e13-712d05592cc5 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.436225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 122/236] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:37.441237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:37.441479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:37.442282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:37.443321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:37.444839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:37.469764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:37.471243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:37.528690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:37.539447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 115/237] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:37.599939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 123/238] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:38.003553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.026950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.061964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.157916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8006d-3ba4-4b9e-b61c-f4dbc5883039 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.220811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32129a61-9ee9-4690-aa34-8a68340677c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.221706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32129a61-9ee9-4690-aa34-8a68340677c1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.222006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32129a61-9ee9-4690-aa34-8a68340677c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.222648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32129a61-9ee9-4690-aa34-8a68340677c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.224741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32129a61-9ee9-4690-aa34-8a68340677c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.227151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32129a61-9ee9-4690-aa34-8a68340677c1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.228832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5c973-1ad0-413d-bd24-ac2d1a24e121 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.229086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 116/239] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:38.230490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d85574-980e-4817-bc50-00562ad0c03d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.230770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d85574-980e-4817-bc50-00562ad0c03d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.231935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d85574-980e-4817-bc50-00562ad0c03d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.233323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 117/240] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:38.241336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d85574-980e-4817-bc50-00562ad0c03d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.272092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31f0f8-a7b8-4496-b579-23aee4238b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.272782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f0f8-a7b8-4496-b579-23aee4238b3f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.273128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f0f8-a7b8-4496-b579-23aee4238b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.273467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f0f8-a7b8-4496-b579-23aee4238b3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.289294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31f0f8-a7b8-4496-b579-23aee4238b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ce31f0f8-a7b8-4496-b579-23aee4238b3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:38.294296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:38.295408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 124/241] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:38.297547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.299268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.299268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.315457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:38.316676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:38.333320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.364946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.390894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d85574-980e-4817-bc50-00562ad0c03d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.391981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 125/242] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:38.393003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.485614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f987357-1c98-4093-8b57-0ee19cac02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.486259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f987357-1c98-4093-8b57-0ee19cac02e6 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.486533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f987357-1c98-4093-8b57-0ee19cac02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.486811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f987357-1c98-4093-8b57-0ee19cac02e6 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.487475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f987357-1c98-4093-8b57-0ee19cac02e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.489763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f987357-1c98-4093-8b57-0ee19cac02e6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.507347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d372-3706-4ae0-9f1c-ca93f5733786 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.546453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.547342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.547805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.548213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.549042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:38.549504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.550997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.560921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 126/243] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:38.567221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.582068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.600838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 118/244] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:38 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:38.647768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393e511-ea9f-4fbd-92f1-8ce2c30de9bc None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.648091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393e511-ea9f-4fbd-92f1-8ce2c30de9bc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.648358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393e511-ea9f-4fbd-92f1-8ce2c30de9bc None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.649755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 119/245] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:38.657758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.658121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.658311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.698936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:38.700265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:38.722883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d13c7-467a-4586-91b8-59d2cc0020c5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.769564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.771344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.771829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.772304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] PATH_INFO: `/v3/users/043df3c215c64d7cb10f126837fd05ac` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.799472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] RBAC: Authorizing `identity:get_user(user_id=043df3c215c64d7cb10f126837fd05ac)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:38.800988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:38.802528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 120/246] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:38.803019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 127/247] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:38 2022] GET /identity/v3/users/043df3c215c64d7cb10f126837fd05ac => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:38.803277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.809169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:38.898594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.905289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:38.938262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e60add-9e19-40d8-8e81-6f9eafcd9e62 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.939215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65c6b-d2bf-4ea7-8166-4bfa64402f55 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:38.985821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae6c8b-7c21-4ffc-a5a4-c393c8764cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.986439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6c8b-7c21-4ffc-a5a4-c393c8764cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.986716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6c8b-7c21-4ffc-a5a4-c393c8764cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.986987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6c8b-7c21-4ffc-a5a4-c393c8764cb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:38.988105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:38.988879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:38.989158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:38.989429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.001742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:39.008644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:39.010088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:39.010879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 121/248] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:39.025445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.081197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae6c8b-7c21-4ffc-a5a4-c393c8764cb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.083370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae6c8b-7c21-4ffc-a5a4-c393c8764cb8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.153400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fe8aa-9d49-4854-be0b-6507c2a23e74 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.171155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.191307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.197830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.198436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.198723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.198995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.199672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:39.199972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.201422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.211264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 128/249] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:44:39.212217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 122/250] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.216415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43230e-2743-4687-a0fd-c35a25f89e4e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.216415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43230e-2743-4687-a0fd-c35a25f89e4e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.216415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43230e-2743-4687-a0fd-c35a25f89e4e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.216415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 129/251] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:39.216415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43230e-2743-4687-a0fd-c35a25f89e4e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.218520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.342122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fab1e-72e0-4d40-8e1e-37821c48596b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.342318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43230e-2743-4687-a0fd-c35a25f89e4e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.378376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.379297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.379577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.379853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.380761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f57e6-d3eb-4d8d-a9f9-4fb3d505222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.381152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.381611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f57e6-d3eb-4d8d-a9f9-4fb3d505222c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.381992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f57e6-d3eb-4d8d-a9f9-4fb3d505222c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.382372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f57e6-d3eb-4d8d-a9f9-4fb3d505222c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.382712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.384693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 123/252] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.385525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.395854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f57e6-d3eb-4d8d-a9f9-4fb3d505222c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6e4f57e6-d3eb-4d8d-a9f9-4fb3d505222c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:39.401363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:39.403524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 130/253] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:39.405608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.412862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.412862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.424117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:39.425485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:39.450572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.454978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.495564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9667ac-24e2-4926-88d4-e9ecf3bf8a27 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.502554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 131/254] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:39.503821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.552266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.553271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.553858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.554316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] PATH_INFO: `/v3/users/97937bcb36684704a0e1c9f7952ab7d3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.555665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] RBAC: Authorizing `identity:get_user(user_id=97937bcb36684704a0e1c9f7952ab7d3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.558162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.559993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 124/255] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/users/97937bcb36684704a0e1c9f7952ab7d3 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.566030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.683458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9db6e2-b175-4d3c-b54b-a067ee316173 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.752023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.753593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.754255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.755756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.756897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] RBAC: Adding query filter params (role.id=c61c993396164231915b06811bea6931, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=043df3c215c64d7cb10f126837fd05ac) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:39.758143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.760439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.766687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 132/256] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/role_assignments?user.id=043df3c215c64d7cb10f126837fd05ac&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=c61c993396164231915b06811bea6931 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.767865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.770552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1473d30-ca26-4e29-bb9c-3ad0055bcf6e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.800303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.808821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.809441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.809717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.809985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.810657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:39.810957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.812396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.812731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:39.821594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 125/257] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.822539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.886090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7363c4f-75be-42f5-83b2-be469d2a4cec None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.925972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.927213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.927701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.928204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.929371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f8948-8c88-4cac-8bd4-202acd009ba1 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:39.929875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:39.931863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:39.934356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 133/258] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:39.935427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:39.967592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:39.968193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:39.968595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:39.969039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:39.980034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:39.985124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:39.986092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:39.986910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 126/259] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:39.987580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.059746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.094347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8facc2-6d5f-4c2f-87ff-69d214c076e4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.113010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.131281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.132041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.132450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.132786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/97937bcb36684704a0e1c9f7952ab7d3/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.133164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfda24-a0cf-47c6-a4c0-d7fbf05d3232 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.134846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] RBAC: Authorizing `identity:create_grant(project_id=cc7140fe2ff44d7a9a15836c34f27dfc, user_id=97937bcb36684704a0e1c9f7952ab7d3, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.136390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.174401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.175050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.175417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.175694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] PATH_INFO: `/v3/users/0cc71147272a472ba402d53f4b4f0503` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.177834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] RBAC: Authorizing `identity:get_user(user_id=0cc71147272a472ba402d53f4b4f0503)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.179540 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.181836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 134/260] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:39 2022] GET /identity/v3/users/0cc71147272a472ba402d53f4b4f0503 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:40.182777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.313321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfad0c-6bd3-4d0b-a15c-1bf0d58beffc None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.351475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.352367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.352804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.353242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.354065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:40.354526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.356248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.365212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 135/261] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:40.366239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.392054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.476065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcdba6-aa07-4890-ba61-7497390882d4 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.510785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 127/262] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:39 2022] PUT /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/97937bcb36684704a0e1c9f7952ab7d3/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:40.513438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.553580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.554512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.555001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.555496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.736037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.738390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.771025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c8fb3-75f0-4a59-bc54-fc0c4ad2cbbe None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:40.792404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.804189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:40.826547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:40.827200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:40.827479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:40.828074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] PATH_INFO: `/v3/users/0cc71147272a472ba402d53f4b4f0503` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:40.829396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] RBAC: Authorizing `identity:get_user(user_id=0cc71147272a472ba402d53f4b4f0503)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:40.831313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:40.834064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 128/263] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/users/0cc71147272a472ba402d53f4b4f0503 => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:40.840557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:40.892489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 136/264] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:40.901355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.007383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639b8b8-0c8f-409c-9562-a9e7eb5d8f34 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.060345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3803f4-19a4-4e99-8039-e305fe31b8b3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.085735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683a6a3-1809-4504-a4ed-fe612f2b2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.086398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683a6a3-1809-4504-a4ed-fe612f2b2b4d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.086627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683a6a3-1809-4504-a4ed-fe612f2b2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.086897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683a6a3-1809-4504-a4ed-fe612f2b2b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.107149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.144137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.145092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.145356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.145438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.153158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.154729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.158283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 137/265] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:41.171078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.192847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.224689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683a6a3-1809-4504-a4ed-fe612f2b2b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.226156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683a6a3-1809-4504-a4ed-fe612f2b2b4d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.369918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 129/266] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:41.370930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482836b-54a2-4d6b-b088-9a73f186b205 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.371214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482836b-54a2-4d6b-b088-9a73f186b205 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.371486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482836b-54a2-4d6b-b088-9a73f186b205 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.372738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 130/267] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:41.380782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f35df-3c11-453e-8afd-dfe16a034519 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.381058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f35df-3c11-453e-8afd-dfe16a034519 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.381322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f35df-3c11-453e-8afd-dfe16a034519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.395714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512da440-2ee0-48e4-a83d-3ffda3ee5b80 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.419321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f94f35df-3c11-453e-8afd-dfe16a034519 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:41.420547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f94f35df-3c11-453e-8afd-dfe16a034519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.456275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.456914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.457215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.457509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.458240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] RBAC: Adding query filter params (role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=0cc71147272a472ba402d53f4b4f0503) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:41.458593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.460018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.472340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 138/268] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:41 2022] GET /identity/v3/role_assignments?user.id=0cc71147272a472ba402d53f4b4f0503&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:41.472340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.549759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 131/269] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:41.551772 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123843d-6a0f-460e-8a05-7278074d74fe None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.552041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123843d-6a0f-460e-8a05-7278074d74fe None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.552329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123843d-6a0f-460e-8a05-7278074d74fe None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.553681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 132/270] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:41.561794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74497605-1260-40fd-928d-9e21239197a0 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.562395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74497605-1260-40fd-928d-9e21239197a0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.562889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74497605-1260-40fd-928d-9e21239197a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.585115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-74497605-1260-40fd-928d-9e21239197a0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:41.587025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-74497605-1260-40fd-928d-9e21239197a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:41.587678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.590636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:41.686254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4a20d-f5c1-4ea0-8d33-12114c0c3bc6 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.702442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 133/271] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:41.735744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.736787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.737752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.738216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.740328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.742166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.746870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 139/272] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:41 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:41.750075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74497605-1260-40fd-928d-9e21239197a0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.880298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74497605-1260-40fd-928d-9e21239197a0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:41.903856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:41.921982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:41.922640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:41.922875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:41.923150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:41.923896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=0cc71147272a472ba402d53f4b4f0503) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:41.924207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:41.925714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:41.939942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 134/273] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:41 2022] GET /identity/v3/role_assignments?user.id=0cc71147272a472ba402d53f4b4f0503&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:42.031733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b64385-3d83-422e-bfb7-ad13c207d1df None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.061580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.076174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.077234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.077705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.078147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.079559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.081221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.095447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 140/274] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:42.095573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.096241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.100947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.226466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63080e-28d9-44da-ad12-bf43ef308baf None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.261409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362785d0-c1e1-41fd-9b3e-ad878b994b77 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.263309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.264051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.265170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.265170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.277884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:42.283041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:42.283986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:42.289391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 141/275] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:42.294538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.312925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.313789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.314070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.314343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.315452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.317365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.362803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.419321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371aba-e3e0-4084-8d15-ff0761efd13d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.447151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 135/276] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:44:42 2022] POST /identity/v3/users => generated 293 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:44:42.468434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.469568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.470047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.470529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.471515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:42.474245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.476061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabebf-12f6-4bb1-a304-56646b5ad7b8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.487417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 142/277] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:42.494634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.566302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.617220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000aa54-7afd-4fc0-a897-ed145bc79282 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.632978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced356a-8cbb-46d0-8abd-c06ebe6d4923 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.633289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced356a-8cbb-46d0-8abd-c06ebe6d4923 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.633539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced356a-8cbb-46d0-8abd-c06ebe6d4923 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.635117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 143/278] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:42.640191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010579-f358-4d5f-9041-30925242efb6 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.640525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010579-f358-4d5f-9041-30925242efb6 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.641131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010579-f358-4d5f-9041-30925242efb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.655368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.656069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.656374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.656654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] PATH_INFO: `/v3/users/f94110618d0949b7916af153a4f1a1d3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.663060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9d010579-f358-4d5f-9041-30925242efb6 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:42.664260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9d010579-f358-4d5f-9041-30925242efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:42.673128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] RBAC: Authorizing `identity:get_user(user_id=f94110618d0949b7916af153a4f1a1d3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.674599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.680353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 136/279] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/users/f94110618d0949b7916af153a4f1a1d3 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:42.682915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:42.750874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 144/280] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:42.796120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e4bff-0b86-4bcb-a5d7-cbe4a20d4200 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:42.813675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:42.832052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:42.834854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.834854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.834854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.834854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, user.id=f94110618d0949b7916af153a4f1a1d3, scope.system=all) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:42.834854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:42.836232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ddb05-3684-458f-ac45-bfbf07893c6c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:42.842018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 137/281] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 1 17:44:42 2022] GET /identity/v3/role_assignments?user.id=f94110618d0949b7916af153a4f1a1d3&scope.system=all&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:42.943457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535c25-5d4c-4b6b-97ce-ec5177c5c3e8 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.943784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535c25-5d4c-4b6b-97ce-ec5177c5c3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.944011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535c25-5d4c-4b6b-97ce-ec5177c5c3e8 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.945754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 145/282] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:42.955034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:42.955351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:42.955580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:42.977552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:42.978797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:43.033069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535c25-5d4c-4b6b-97ce-ec5177c5c3e8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.055038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 138/283] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:43.082364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.142812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535c25-5d4c-4b6b-97ce-ec5177c5c3e8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.182981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.183984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.184492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.184963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.205565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:43.210833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:43.211734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:43.212581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:43.215193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 146/284] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:43.221117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.376754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5f5c2-5668-4dde-98c7-e3751117c420 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.393931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.425984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.427367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.427880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.428490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.429162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:43.429464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.430834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.441336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 139/285] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:43.447041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.447668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.567364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aed3f-e610-4731-a428-a5ca21e3d035 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.569288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.613192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.614767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.615315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.615813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] PATH_INFO: `/v3/users/043df3c215c64d7cb10f126837fd05ac` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.617246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] RBAC: Authorizing `identity:get_user(user_id=043df3c215c64d7cb10f126837fd05ac)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.619072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.621248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 147/286] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/users/043df3c215c64d7cb10f126837fd05ac => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:43.627291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.670427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384ef2f-a4c7-4477-b6ac-ab801150e47a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.740006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0fbac-ab5f-4818-914c-e382b75046ff None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.775148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.775895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.776181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.776495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.779357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.780576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.781054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.781522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.783209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.784870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.787718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 148/287] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:43.793540 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.796125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:43.801972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:43.802881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:43.803697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:43.812785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 140/288] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:43.818009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:43.839256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.890532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:43.901874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679f57-cea8-47ca-857c-7a799714d79e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:43.938017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:43.939033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:43.939551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:43.940022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/043df3c215c64d7cb10f126837fd05ac/roles/c61c993396164231915b06811bea6931` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:43.942273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] RBAC: Authorizing `identity:create_grant(project_id=cc7140fe2ff44d7a9a15836c34f27dfc, user_id=043df3c215c64d7cb10f126837fd05ac, role_id=c61c993396164231915b06811bea6931)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:43.944049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:43.987667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115b60e-c317-4fb7-8ecd-e2c880fd67c5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.024774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 149/289] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:44:43 2022] PUT /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/users/043df3c215c64d7cb10f126837fd05ac/roles/c61c993396164231915b06811bea6931 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:44.034948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.036674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.037338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.037816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.038757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:44.039363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.041071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.060543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 141/290] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:44.066537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.204292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:44.212653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4124da-6972-4436-bc43-529f5f277d1e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.262232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.262872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.263147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.263417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] PATH_INFO: `/v3/users/97937bcb36684704a0e1c9f7952ab7d3` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.269144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] RBAC: Authorizing `identity:get_user(user_id=97937bcb36684704a0e1c9f7952ab7d3)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.271202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.272933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 150/291] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:44 2022] GET /identity/v3/users/97937bcb36684704a0e1c9f7952ab7d3 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:44.278658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.415669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5589d17-d9ab-4eb5-9492-a8879950987e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.424339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:44.462123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.462676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.462957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.463232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.552427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.553902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.659374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:44.670759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 142/292] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:44.672414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.783497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273854e-16a9-4732-ac05-e8662985128b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.818997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b4bb3-e73d-4de1-bd2e-6f17cc39b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.819630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b4bb3-e73d-4de1-bd2e-6f17cc39b8af None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.819908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b4bb3-e73d-4de1-bd2e-6f17cc39b8af None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.820183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b4bb3-e73d-4de1-bd2e-6f17cc39b8af None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.821345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b4bb3-e73d-4de1-bd2e-6f17cc39b8af None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.822818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b4bb3-e73d-4de1-bd2e-6f17cc39b8af None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.824488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 151/293] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:44 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:44.830292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:44.950937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aee237-acf1-4682-85a1-a33cd7b0557a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:44.988079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:44.989346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:44.989632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:44.989910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:44.990649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=97937bcb36684704a0e1c9f7952ab7d3) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:44.990966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:44.992529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9733a3-ca2e-443c-847c-d9f13b17f98f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:44.998946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 143/294] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:44 2022] GET /identity/v3/role_assignments?user.id=97937bcb36684704a0e1c9f7952ab7d3&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:45.013275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3887d-305c-4651-a1ac-5f80c92d851a None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.013575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3887d-305c-4651-a1ac-5f80c92d851a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.013831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3887d-305c-4651-a1ac-5f80c92d851a None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.015467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 152/295] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:45.021654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422017-baf2-4b9c-b970-b228c1bae1b9 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.021932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422017-baf2-4b9c-b970-b228c1bae1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.022210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422017-baf2-4b9c-b970-b228c1bae1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.042828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5d422017-baf2-4b9c-b970-b228c1bae1b9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:45.043991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5d422017-baf2-4b9c-b970-b228c1bae1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:45.120281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:45.135307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 144/296] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:45.148442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a266ff-15de-409a-9eb9-e73f817ead05 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.148442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a266ff-15de-409a-9eb9-e73f817ead05 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.148442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a266ff-15de-409a-9eb9-e73f817ead05 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.148442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 153/297] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:45.151115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bc06d-cb0b-48f2-a6f1-4e28c3302381 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.151404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bc06d-cb0b-48f2-a6f1-4e28c3302381 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.151742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bc06d-cb0b-48f2-a6f1-4e28c3302381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.171945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ee8bc06d-cb0b-48f2-a6f1-4e28c3302381 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:45.173191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ee8bc06d-cb0b-48f2-a6f1-4e28c3302381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:45.249144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 145/298] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:45.329879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42400-d879-4b01-8a22-3a2ca9770be1 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.330176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42400-d879-4b01-8a22-3a2ca9770be1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.330365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42400-d879-4b01-8a22-3a2ca9770be1 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.331949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 154/299] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:45.338032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.338484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.338652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.359914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:45.361135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:45.426171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42400-d879-4b01-8a22-3a2ca9770be1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:45.432950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 146/300] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:45.468372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:45.541611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42400-d879-4b01-8a22-3a2ca9770be1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:45.547813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:45.589852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:45.591728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.592233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.592784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.594106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:45.596263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:45.599636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 155/301] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:45.605248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:45.607105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:45.665130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01c494-a1ed-47fb-9ea9-6c0e6fb53260 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:45.704996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:45.705815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.706196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.706568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.718397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:45.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:45.725121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:45.726161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:45.727753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 147/302] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:45.733796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:45.744064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b4e66-328d-414d-9c02-210825ee4a70 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:45.746364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:45.781745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:45.782363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.782647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.782952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] PATH_INFO: `/v3/groups` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.783593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:45.785037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:45.837934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daff0a-877d-45b3-a345-3e19c1de2e86 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:45.846881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:45.863367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 156/303] 10.222.0.15 () {60 vars in 1134 bytes} [Thu Sep 1 17:44:45 2022] POST /identity/v3/groups => generated 229 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:45.864872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:45.886766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:45.887536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:45.887888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:45.888279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:45.889105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:45.889482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:45.892666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:45.909838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 148/304] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:45.911767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:46.005320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc3e7c-7058-4fe0-8a19-550f5e1f437a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:46.040515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef5688-84a8-4c76-8733-4f9b1314cfc7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:46.057973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:46.058961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.059242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.059517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.073890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:46.080272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:46.081213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:46.082012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 157/305] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:46.088223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:46.089043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.089385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.089718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] PATH_INFO: `/v3/users/87d933ff904348108bbd15a61a6c63dc` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.091168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:46.107503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] RBAC: Authorizing `identity:get_user(user_id=87d933ff904348108bbd15a61a6c63dc)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:46.109815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:46.116335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 149/306] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:45 2022] GET /identity/v3/users/87d933ff904348108bbd15a61a6c63dc => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:46.121169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:46.123252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:46.153572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:46.228934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e176f2-db85-4f80-a3a2-97425fd50fc0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:46.270059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d37ba-4916-4b33-86b8-46eb183498e5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:46.275947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:46.276720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.277070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.277417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.278194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:46.278585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:46.280684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:46.296403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 158/307] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:46.300613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:46.316161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:46.318284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.319023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.319495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.320738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] RBAC: Adding query filter params (role.id=c61c993396164231915b06811bea6931, user.id=87d933ff904348108bbd15a61a6c63dc, scope.system=all) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:46.321721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:46.323952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:46.331007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 150/308] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 1 17:44:46 2022] GET /identity/v3/role_assignments?user.id=87d933ff904348108bbd15a61a6c63dc&scope.system=all&role.id=c61c993396164231915b06811bea6931 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:46.407154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:46.427023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c910-e5ac-4e4b-b46e-74212f9dfe20 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:46.489162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee10e83-786f-4e27-88bb-fcbc85dc0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:46.489162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10e83-786f-4e27-88bb-fcbc85dc0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.489162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10e83-786f-4e27-88bb-fcbc85dc0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.489162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10e83-786f-4e27-88bb-fcbc85dc0ef0 None admin] PATH_INFO: `/v3/users/f94110618d0949b7916af153a4f1a1d3` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.489510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee10e83-786f-4e27-88bb-fcbc85dc0ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=f94110618d0949b7916af153a4f1a1d3)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:46.489510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee10e83-786f-4e27-88bb-fcbc85dc0ef0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:46.489510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 159/309] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:46 2022] GET /identity/v3/users/f94110618d0949b7916af153a4f1a1d3 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:46.489707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:46.542241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2e509-808d-4856-9553-b9bd0903a044 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.542525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2e509-808d-4856-9553-b9bd0903a044 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.542805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2e509-808d-4856-9553-b9bd0903a044 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.544603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 160/310] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:46.553132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.554528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.554528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.577902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:46.579299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:46.607124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:46.616775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7caae-1190-4133-9744-3dd13e41893d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:46.659922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:46.661830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:46.662273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:46.662711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] PATH_INFO: `/v3/system/users/f94110618d0949b7916af153a4f1a1d3/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:46.664911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f94110618d0949b7916af153a4f1a1d3, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:46.666740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:46.680361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 161/311] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:46.756194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 151/312] 10.222.0.15 () {58 vars in 1334 bytes} [Thu Sep 1 17:44:46 2022] PUT /identity/v3/system/users/f94110618d0949b7916af153a4f1a1d3/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:46.965009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:47.057221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:47.076269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33f8b7-d204-4de3-a6f4-a24655956b9a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:47.112915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:47.113945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.114434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.114889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.125550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:47.130675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:47.131596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:47.132452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:47.134360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 162/313] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:47.143044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:47.247663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869136f-47c0-4c52-b82e-06d85521843f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:47.284108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:47.284749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.285023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.285891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.286522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:47.286924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:47.288351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:47.297773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 152/314] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:47.304030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:47.447724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2091a-5c1b-40f6-987c-689c64a3dc1d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:47.483636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb8eb3-b390-48be-ae65-ad5adb014fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:47.484404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb8eb3-b390-48be-ae65-ad5adb014fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.484757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb8eb3-b390-48be-ae65-ad5adb014fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb8eb3-b390-48be-ae65-ad5adb014fa8 None admin] PATH_INFO: `/v3/users/043df3c215c64d7cb10f126837fd05ac` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.486298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb8eb3-b390-48be-ae65-ad5adb014fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=043df3c215c64d7cb10f126837fd05ac)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:47.487798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb8eb3-b390-48be-ae65-ad5adb014fa8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:47.490486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 163/315] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:47 2022] GET /identity/v3/users/043df3c215c64d7cb10f126837fd05ac => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:47.501674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:47.539263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:47.674758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa657d2-dc32-47cf-84ce-806dc9b49702 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:47.723714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:47.725952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.727782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.728954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.861558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:47.867801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:47.917054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:47.928742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5863d-1e6e-49b6-9421-9d361ec25e63 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.937665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5863d-1e6e-49b6-9421-9d361ec25e63 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.937665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5863d-1e6e-49b6-9421-9d361ec25e63 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.937665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 164/316] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:47.937665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:47.937665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:47.937665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:47.978122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:47.980752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:47.988185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:48.044289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 153/317] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:48.049763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.073907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 165/318] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:48.153813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec6b8e-064a-45a3-8734-9ce497cf8b7e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.172431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:48.189779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.190506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.190781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.191049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.192213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.193676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.195659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 154/319] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:48 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:48.201722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.320646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8bcd7-1d26-4b31-acdf-f3fc40e90d9e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.346902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.359046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.360003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.360538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.360813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.361557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] RBAC: Adding query filter params (role.id=c61c993396164231915b06811bea6931, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc, user.id=043df3c215c64d7cb10f126837fd05ac) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:48.361864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.363522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f99ceb-8e3c-417a-8690-8516b965d1f8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.370292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 166/320] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:44:48 2022] GET /identity/v3/role_assignments?user.id=043df3c215c64d7cb10f126837fd05ac&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=c61c993396164231915b06811bea6931 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:48.463499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cded0d-e3af-44b4-bd9c-7a4d81c12d4a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.481943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1895b-e683-453e-9691-8f3eb672bafe None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.482652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1895b-e683-453e-9691-8f3eb672bafe None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.483248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1895b-e683-453e-9691-8f3eb672bafe None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.485400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 167/321] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:48.493022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02622a62-1998-4705-a541-62565abc46d5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.493348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02622a62-1998-4705-a541-62565abc46d5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.493576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02622a62-1998-4705-a541-62565abc46d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.501178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:48.509829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.511208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.511665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.512081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.513608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.515205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.517521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 155/322] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:48.520984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-02622a62-1998-4705-a541-62565abc46d5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:48.522855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-02622a62-1998-4705-a541-62565abc46d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:48.524426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.603696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:48.612388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 168/323] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:48.657999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20dd9-ac99-4f3f-8937-60c8f5f26cf7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:48.696074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:48.696717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.696994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.697264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] PATH_INFO: `/v3/groups` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.697893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:48.699308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:48.735020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1075d-690f-4ebb-8c91-da17800ec1dc None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.735187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1075d-690f-4ebb-8c91-da17800ec1dc None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.735382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1075d-690f-4ebb-8c91-da17800ec1dc None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.736764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 169/324] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:48.741505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:48.751177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:48.751177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:48.760059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:48.761248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:48.805341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 156/325] 10.222.0.15 () {60 vars in 1134 bytes} [Thu Sep 1 17:44:48 2022] POST /identity/v3/groups => generated 226 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:44:48.829153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 170/326] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:48.885418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:48.998081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392580-9c77-45d1-b69f-294181bf9a23 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.013398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.035659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667823e-813f-4869-8295-ec77d868991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.036286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667823e-813f-4869-8295-ec77d868991c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.036608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667823e-813f-4869-8295-ec77d868991c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.036874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667823e-813f-4869-8295-ec77d868991c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.049774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667823e-813f-4869-8295-ec77d868991c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-0667823e-813f-4869-8295-ec77d868991c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:49.055419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:49.061726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:49.062530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 157/327] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:49.063206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.103399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667823e-813f-4869-8295-ec77d868991c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.131832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.182098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92008-1883-4497-bf48-1a89927302ec None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.225528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.226154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.226489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.226764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.227441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:49.227742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.229267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.244859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 171/328] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:49.247577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667823e-813f-4869-8295-ec77d868991c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.252984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.291874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.293131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.293629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.294250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.310489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:49.318350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:49.319468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:49.320531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:49.323389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 158/329] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:44:49.329055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.383054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.388020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4893-dc7a-409a-9804-4e419d345b7f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.431726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.445721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.446690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.447125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.447551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] PATH_INFO: `/v3/users/87d933ff904348108bbd15a61a6c63dc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.449566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] RBAC: Authorizing `identity:get_user(user_id=87d933ff904348108bbd15a61a6c63dc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.451731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.454663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 172/330] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/users/87d933ff904348108bbd15a61a6c63dc => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:49.462890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.492035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91efb3f-8b8c-4889-882a-a943b7e78d1a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.554279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.554878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.555148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.555815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.557284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:49.557753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.559554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.571681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 159/331] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:49.577303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.601614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a563ba4-c547-4a78-91a4-a5d5bb1cf6df None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.642105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.642776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.643054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.643325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] PATH_INFO: `/v3/system/users/87d933ff904348108bbd15a61a6c63dc/roles/c61c993396164231915b06811bea6931` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.645578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=87d933ff904348108bbd15a61a6c63dc, role_id=c61c993396164231915b06811bea6931)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.647020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.699767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.705976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd5bdc-9a0f-4ece-918d-cf9954a70bb2 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.710253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 173/332] 10.222.0.15 () {58 vars in 1334 bytes} [Thu Sep 1 17:44:49 2022] PUT /identity/v3/system/users/87d933ff904348108bbd15a61a6c63dc/roles/c61c993396164231915b06811bea6931 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:49.745132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a844-2809-440a-a3d0-6e7c85111d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.745990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a844-2809-440a-a3d0-6e7c85111d4f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.746681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a844-2809-440a-a3d0-6e7c85111d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.746956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a844-2809-440a-a3d0-6e7c85111d4f None admin] PATH_INFO: `/v3/users/f94110618d0949b7916af153a4f1a1d3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.748272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825a844-2809-440a-a3d0-6e7c85111d4f None admin] RBAC: Authorizing `identity:get_user(user_id=f94110618d0949b7916af153a4f1a1d3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.749830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825a844-2809-440a-a3d0-6e7c85111d4f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.751903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 160/333] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/users/f94110618d0949b7916af153a4f1a1d3 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:49.758742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:49.826802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:49.872587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafcf2a-c0a8-4ef2-b546-aa01edae8c71 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:49.910227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:49.910893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:49.911380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:49.911656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:49.912417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] RBAC: Adding query filter params (role.id=6bc7a311754c40b2bbf21dccc998bc9b, user.id=f94110618d0949b7916af153a4f1a1d3, scope.system=all) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:49.912723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:49.914210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28db775-3c7f-4e0b-b71d-364ced7ffe8b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:49.927397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 174/334] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 1 17:44:49 2022] GET /identity/v3/role_assignments?user.id=f94110618d0949b7916af153a4f1a1d3&scope.system=all&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:51.079453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d07e-2d93-41be-a07d-0151de2b79da None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.080457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d07e-2d93-41be-a07d-0151de2b79da None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.081016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d07e-2d93-41be-a07d-0151de2b79da None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.083136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 161/335] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:51.088664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5076-8ecf-46f7-88f0-05e63bd5ec16 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.089507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5076-8ecf-46f7-88f0-05e63bd5ec16 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.089982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df5076-8ecf-46f7-88f0-05e63bd5ec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.110190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-54df5076-8ecf-46f7-88f0-05e63bd5ec16 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:51.111710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-54df5076-8ecf-46f7-88f0-05e63bd5ec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:51.194521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 175/336] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:51.418486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fdfb-8836-42f3-a53f-93e5ed5c96d5 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.418812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fdfb-8836-42f3-a53f-93e5ed5c96d5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.419070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fdfb-8836-42f3-a53f-93e5ed5c96d5 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.420591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 162/337] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:51.426523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.427035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.427507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.450146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:51.451654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:51.505257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:51.530890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 176/338] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:51.762869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643fdfb-8836-42f3-a53f-93e5ed5c96d5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:51.889136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643fdfb-8836-42f3-a53f-93e5ed5c96d5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:51.928574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:51.929224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:51.929489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:51.929755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:51.945210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:51.950835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:51.951710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:51.952550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 163/339] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:51.958108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.074713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.120904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.138349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692beb64-0fb9-4e97-bafa-14cc716f2314 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.224935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.225558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.225838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.226112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.226794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:52.227095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.232098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.256795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 177/340] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:52.262446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.303910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92f15-31e3-4444-a7a5-d44f3fa55d13 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.334295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.366671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59190a08-30e0-4aac-8bac-2f879ebb8383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.367309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59190a08-30e0-4aac-8bac-2f879ebb8383 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.367697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59190a08-30e0-4aac-8bac-2f879ebb8383 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.367866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59190a08-30e0-4aac-8bac-2f879ebb8383 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.386698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59190a08-30e0-4aac-8bac-2f879ebb8383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-59190a08-30e0-4aac-8bac-2f879ebb8383 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:52.392452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:52.393525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:52.394369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 164/341] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:52.395661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fb110-12e4-4ffe-8801-57596774991f None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.395821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fb110-12e4-4ffe-8801-57596774991f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.396208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fb110-12e4-4ffe-8801-57596774991f None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.397739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 165/342] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:52.403713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fb110-12e4-4ffe-8801-57596774991f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.444835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e48a1-5fd6-482f-a5f4-93f7c7133d92 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.495409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8ab8f-29ab-441c-9ebc-33f629f4ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.496616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8ab8f-29ab-441c-9ebc-33f629f4ba40 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.496954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8ab8f-29ab-441c-9ebc-33f629f4ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.497520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8ab8f-29ab-441c-9ebc-33f629f4ba40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.542443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.568824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fb110-12e4-4ffe-8801-57596774991f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.600180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8ab8f-29ab-441c-9ebc-33f629f4ba40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.602911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8ab8f-29ab-441c-9ebc-33f629f4ba40 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.621040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.622057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.622649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.623101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.624080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:52.624619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.626200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d055606-615f-40c1-b2f7-0c4c247a78dd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.638480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 166/343] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:52.640032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.640562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.642372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.663374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:52.665214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.665479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:52.748593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 178/344] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:52.749837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c920-604d-447c-a00a-dadd4d65f2ab None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.750124 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c920-604d-447c-a00a-dadd4d65f2ab None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.750226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 167/345] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:52.750401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c920-604d-447c-a00a-dadd4d65f2ab None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.751033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.751701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 179/346] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:52.752411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c920-604d-447c-a00a-dadd4d65f2ab None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:52.846323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.872580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34f277-58f4-4c58-9ed7-61e67a58e22f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.911721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.912655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.913111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.913578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.931677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c920-604d-447c-a00a-dadd4d65f2ab None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:52.968864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea332ed4-3668-489c-985e-23f0d3414364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:52.969561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea332ed4-3668-489c-985e-23f0d3414364 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.969884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea332ed4-3668-489c-985e-23f0d3414364 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.970163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea332ed4-3668-489c-985e-23f0d3414364 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.971288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea332ed4-3668-489c-985e-23f0d3414364 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.972836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea332ed4-3668-489c-985e-23f0d3414364 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.974513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 180/347] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:52.976040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:52.976040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:52.976219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:52.994308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:52.994667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:52.996733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:52.997189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:52.998369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:53.079449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 181/348] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:53.079449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.102275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.167806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 168/349] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:53.174744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.272342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a6a5-13e4-46fd-bba8-f68d9445ebb0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.338005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.338644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.338921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.339191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.344631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.352112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.353589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.355250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 182/350] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:53.356010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.377470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bae1b7-f588-42e5-b086-e141841b7750 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.444066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.444647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.444923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.445188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.447096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.448557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.453532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 169/351] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:53.454325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.513608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdc76e-ccc9-4545-9668-489934aae666 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.514960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.567822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.568607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.568946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.569324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.584070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:53.586157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a9949-6094-4f08-82d9-9a5aa426a9a2 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:53.590862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:53.591809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 183/352] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:53.593287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.632144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.632144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.632144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.632144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.632617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=7de433353c124372b395806b9f1eb576) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:53.632617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.633747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.640544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 170/353] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:53.641611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.666999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.729492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07679755-8c14-4334-ade2-491c8724ddb7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.776274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.778373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.778373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.778373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.781452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.783179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.787108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 184/354] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:53.787108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.793730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4fa88-c689-4830-a6ba-3c0932636ce0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.809698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.847495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710353e-8f3a-40cc-8b7a-f841f637f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.851613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710353e-8f3a-40cc-8b7a-f841f637f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.851891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710353e-8f3a-40cc-8b7a-f841f637f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.852160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710353e-8f3a-40cc-8b7a-f841f637f0e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.865788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710353e-8f3a-40cc-8b7a-f841f637f0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-a710353e-8f3a-40cc-8b7a-f841f637f0e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:53.871674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:53.872641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 17:44:53.873473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:53.879527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 171/355] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:53.881124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07cfa-4baa-43e0-874d-854246dae493 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.881417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07cfa-4baa-43e0-874d-854246dae493 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.881690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07cfa-4baa-43e0-874d-854246dae493 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.883246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 172/356] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:53.885213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f1143-662e-4f85-b5cc-884ba009b493 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.885494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f1143-662e-4f85-b5cc-884ba009b493 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.885767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f1143-662e-4f85-b5cc-884ba009b493 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.887068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 173/357] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:53.887822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f1143-662e-4f85-b5cc-884ba009b493 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:53.916264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedd4d-b1a4-4992-bae0-b22bce962442 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:53.946872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:53.956046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:53.957633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:53.966964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:53.966964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:53.966964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:53.966964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:53.966964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:53.974018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 185/358] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:53.975234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.019110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f1143-662e-4f85-b5cc-884ba009b493 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.064086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.064725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.065462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.065739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.067024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=7de433353c124372b395806b9f1eb576) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:54.067383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:54.069324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223122ba-b057-42f2-be15-d595e0df0c27 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:54.078625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 174/359] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:54.080058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.080365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.080642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.099807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0269-cc67-4101-a199-c1da3bb9af5a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.104137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:54.105463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:54.141508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:54.149385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.150618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.151549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.152287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.153532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:54.162040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:54.162040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e362-a020-4ef1-aea5-1329db714637 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:54.168433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 186/360] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:54.169984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.170406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.170784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.194232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 175/361] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:54.194774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:54.195110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.196156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:54.290620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:54.319964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 187/362] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:54.321404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.353886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e43a9-6b17-4e50-afbf-ce8af5b8dca1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.417413 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.418272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.418721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.419133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.476433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:54.507899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06f10-51bd-4235-9518-e9bb7d143145 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.558980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.560168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.560697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.561115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] PATH_INFO: `/v3/users/87d933ff904348108bbd15a61a6c63dc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.562479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] RBAC: Authorizing `identity:get_user(user_id=87d933ff904348108bbd15a61a6c63dc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:54.564866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:54.567456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 188/363] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/users/87d933ff904348108bbd15a61a6c63dc => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:54.572113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:54.572113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:54.573635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.663387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:54.751691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5f0cf-001f-4b1e-b38f-24dff73f2845 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.798246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 176/364] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:54.801328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.828621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.829165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.829448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.829725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.830470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] RBAC: Adding query filter params (role.id=c61c993396164231915b06811bea6931, user.id=87d933ff904348108bbd15a61a6c63dc, scope.system=all) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:54.830780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:54.837966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:54.851555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 189/365] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/role_assignments?user.id=87d933ff904348108bbd15a61a6c63dc&scope.system=all&role.id=c61c993396164231915b06811bea6931 => generated 510 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:54.852546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:54.883176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:54.929886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ae8de-ae2a-4563-afc5-a69419e377e4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:54.967965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:54.968734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:54.969077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:54.969416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:54.982049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:54.987377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:54.988661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:44:54.989693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:54.990907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 177/366] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:54.991916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.072356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b757d2-f6df-4bad-8ae3-e6ee80ef40ae None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.091305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.123467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30dd76-9055-4907-9bda-52642f8612d8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.146107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.146809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.147105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.147484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.149426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.151775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.155366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 190/367] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:55.156143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.170319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.190859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.192171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.192792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.193065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.213215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:55.218178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:55.219063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:55.219884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 178/368] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:55.221367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.280913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f68f4f-18a9-4f0e-8598-0827dd8a1074 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.329652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.331048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.331524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.332026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.332912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:55.333407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.334961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.344304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 191/369] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:55.345381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.384709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106bf7e-abf8-4de2-a16a-2a15872d1c88 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.409066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.425104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.427168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.427612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.428298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.429851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.432073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.434842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 179/370] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:55 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:55.435921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.493585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735c0ab-6c88-40f5-a1e7-d6dd896d512e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.501278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.540569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.542131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.542585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.543005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.544188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:55.544672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.547085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.560055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 192/371] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:55.565230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.571508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca041aa9-eb36-468c-bc54-0ad99c84d388 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.616688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16beb687-8ff2-4fe2-a391-132226f9664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.617751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16beb687-8ff2-4fe2-a391-132226f9664d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.618536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16beb687-8ff2-4fe2-a391-132226f9664d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.618957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16beb687-8ff2-4fe2-a391-132226f9664d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.679166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.682799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16797fd1-4020-49cd-82e1-5b6a96134b3b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.702970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16beb687-8ff2-4fe2-a391-132226f9664d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.704671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16beb687-8ff2-4fe2-a391-132226f9664d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.730859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.731565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.731913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.733050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.733857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:55.737352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:55.739829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:55.748857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 193/372] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:44:55 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:55.749914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:55.793628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:55.891800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 180/373] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:55.895222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc51228-2f01-464a-bae6-1c3636966871 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.895222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc51228-2f01-464a-bae6-1c3636966871 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.895222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc51228-2f01-464a-bae6-1c3636966871 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.896152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 181/374] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:55.898769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf8ff3-65bc-41e2-b9af-f5167c18b328 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:55.901777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.902058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.902332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:55.924182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:55.925418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:55.956521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:55.958497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:55.959287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:55.959766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.010078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 182/375] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:56.011643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.011759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.055167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.057164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.175865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 194/376] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:44:56.182002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.235440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.266498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.292935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9c80f-ee23-4fed-bcff-299add16f2ea None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.318492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa58eb-5ae0-4dba-9025-ba609e7cb385 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.362198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.364195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.364729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.365225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.365903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.366615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.366895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.367142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.367250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] PATH_INFO: `/v3/projects/195190be58ff49f7a586c1baca5891d6` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.368285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] RBAC: Authorizing `identity:get_project(project_id=195190be58ff49f7a586c1baca5891d6)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.369100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.369722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.371502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 195/377] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:56.376649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.376769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 183/378] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/projects/195190be58ff49f7a586c1baca5891d6 => generated 344 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:56.382234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.517516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a2c40-f90c-485d-8cbd-455afb47b910 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.517930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82029dd-3008-4b0f-86a7-53025e860abd None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.557743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.558650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.559291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.559565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.559839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] PATH_INFO: `/v3/groups/8747358a89d442a0b2a8df8ad7d4da2c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.564371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.564973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.565525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.565806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.567027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] RBAC: Authorizing `identity:get_group(group_id=8747358a89d442a0b2a8df8ad7d4da2c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.567506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.568575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.569014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.570838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 196/380] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:56.571621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.576035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 184/380] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/groups/8747358a89d442a0b2a8df8ad7d4da2c => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:56.581202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.605343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.708185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af439d8f-7bd3-4e94-9128-a8fba69b7058 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.709053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ad1ae-3e58-4012-bd54-834c4095b5b9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.747440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.747915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.748187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.748510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.749238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:56.749545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.750990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.756489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce788bc-f838-43c7-bfb0-9ed012d28552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.757316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce788bc-f838-43c7-bfb0-9ed012d28552 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.757738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce788bc-f838-43c7-bfb0-9ed012d28552 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.758048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 185/381] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:56.758453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce788bc-f838-43c7-bfb0-9ed012d28552 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.758870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.775878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce788bc-f838-43c7-bfb0-9ed012d28552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-dce788bc-f838-43c7-bfb0-9ed012d28552 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:56.782351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:56.783345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:56.784233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:44:56.786895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 197/382] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:56.791785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb522c3-d4b1-4d97-9688-3f11f1d18b78 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.791785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb522c3-d4b1-4d97-9688-3f11f1d18b78 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.791785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb522c3-d4b1-4d97-9688-3f11f1d18b78 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.791785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 198/383] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:56.793535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb522c3-d4b1-4d97-9688-3f11f1d18b78 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:56.895189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.895382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:56.930684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade0cf8-241c-4f07-9fef-b24d2d9b5c1d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.953180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb522c3-d4b1-4d97-9688-3f11f1d18b78 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:56.972362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:56.973550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.973824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.974098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:56.974842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] RBAC: Adding query filter params (group.id=8747358a89d442a0b2a8df8ad7d4da2c, role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=195190be58ff49f7a586c1baca5891d6) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:56.975152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:56.977785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78392d-4aea-4bfe-a493-b727285435f3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:56.983869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 186/384] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/role_assignments?group.id=8747358a89d442a0b2a8df8ad7d4da2c&scope.project.id=195190be58ff49f7a586c1baca5891d6&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:56.985238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:56.985525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:56.985800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.004957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.005609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.006043 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.006319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.006894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:57.007000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:57.007344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.008235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:57.008806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.021436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 199/385] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:57.029665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.175589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620d65-f935-4fc9-826f-77aa784eec1b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:57.206178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 187/386] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:57.212770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:57.224937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.234290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.234290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.234290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.234290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.234290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.234290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 200/387] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:57 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:57.237083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.313116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:57.346519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c65b8-212a-4cd0-81f0-27c6f0544ba3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:57.384167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.384872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.385150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.385418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.386651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.388077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.392424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 188/388] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:57 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:57.396409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.502529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebcfc7-c76a-4501-8eb6-1e71521d3747 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:57.543472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dd8ef-978a-44ac-bc44-acadf31456cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:57.544141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd8ef-978a-44ac-bc44-acadf31456cb None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.544446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd8ef-978a-44ac-bc44-acadf31456cb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.544723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd8ef-978a-44ac-bc44-acadf31456cb None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dd8ef-978a-44ac-bc44-acadf31456cb None admin] RBAC: Authorizing `identity:create_grant(project_id=7de433353c124372b395806b9f1eb576, group_id=1682db9ef7bd421aa7465a458e0f8b26, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:57.548129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dd8ef-978a-44ac-bc44-acadf31456cb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:57.654076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 201/389] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 1 17:44:57 2022] PUT /identity/v3/projects/7de433353c124372b395806b9f1eb576/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:44:57.847330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:57.994492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4ee0-ca1f-49d4-91e5-86356939a16a None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:57.994823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4ee0-ca1f-49d4-91e5-86356939a16a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:57.994975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4ee0-ca1f-49d4-91e5-86356939a16a None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:57.996639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 202/390] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:58.002772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.002772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.002772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.031488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:58.032758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:58.038978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1001-ba0b-475a-8639-3fc5014fb7b3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.081377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.082552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.082826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.083096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.095293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.101103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:58.106373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:58.119257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:58.121195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 189/391] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:44:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:58.122824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.122965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 203/392] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:44:58.245768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89894b-5f17-4e72-9b43-d833d6fa190f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.285681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.286383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.286665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.286941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.287618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:58.287920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.289316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.298480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 190/393] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:58.304512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.424709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b58c6c-35fc-4607-82ad-57e590293c57 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.466226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.467161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.467617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.468069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.469394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.471008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.473010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 204/394] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:44:58.481398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.547260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.594539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52c09-d1e3-4061-8ae1-659b46ec02d0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.638954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.639597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.639886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.640156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.641601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.643035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.645626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 191/395] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:58.651136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.688673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:58.707613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.813989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e56e2-904c-4325-9cd1-2dea5517cb0b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.815898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc1b2-b594-44ce-92b3-b3be66049b21 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:58.861599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.862488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.862778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.863759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.866903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] RBAC: Authorizing `identity:create_grant(project_id=7de433353c124372b395806b9f1eb576, group_id=1682db9ef7bd421aa7465a458e0f8b26, role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:58.868539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:58.868997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:58.890656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:58.891573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:58.891859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:58.892137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:58.905047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:44:58.910445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:44:58.911763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:44:58.912834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 192/396] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:44:58.918427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.001438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 205/397] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 1 17:44:58 2022] PUT /identity/v3/projects/7de433353c124372b395806b9f1eb576/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:59.072160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016ec2-18cf-4f5f-b752-00199da344b4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.110294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.111193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.111666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.112094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.113868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:44:59.114355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.115909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.126317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 193/398] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:44:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:59.133750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.242728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c030-2b7b-4d81-91ff-26b6cbc17025 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.272716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:59.293988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.294625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.294906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.295181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.296368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.297881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.299865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 206/399] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:44:59 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:59.308696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.411850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:59.487125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc2f3-e511-4a91-84a4-0afbbfe2d91b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.536079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34891f29-d6be-4367-9cea-cb81c1cadca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.536970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34891f29-d6be-4367-9cea-cb81c1cadca3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.537388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34891f29-d6be-4367-9cea-cb81c1cadca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.537805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34891f29-d6be-4367-9cea-cb81c1cadca3 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.539093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34891f29-d6be-4367-9cea-cb81c1cadca3 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.540689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34891f29-d6be-4367-9cea-cb81c1cadca3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.542451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 194/400] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:44:59 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:44:59.548338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:44:59.612213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36e3c-8b0a-4543-bd45-23eabfa42fec None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.612773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36e3c-8b0a-4543-bd45-23eabfa42fec None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.613225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36e3c-8b0a-4543-bd45-23eabfa42fec None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.614974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 195/401] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:44:59.620380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f21d5d-9610-4b53-baf7-a89f6d71710e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.620935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f21d5d-9610-4b53-baf7-a89f6d71710e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.621401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f21d5d-9610-4b53-baf7-a89f6d71710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.640120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-51f21d5d-9610-4b53-baf7-a89f6d71710e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:59.641910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-51f21d5d-9610-4b53-baf7-a89f6d71710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:59.720062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 196/402] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:44:59.736691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704abb7-9802-48b0-9df8-f2d5f8c96707 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:44:59.773165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:44:59.773827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.774104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.774378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.776288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] RBAC: Authorizing `identity:create_grant(project_id=cc7140fe2ff44d7a9a15836c34f27dfc, group_id=1682db9ef7bd421aa7465a458e0f8b26, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:44:59.777739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:44:59.814257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8a8d8-d6ec-41c8-bc86-189c0a5c46fd None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.814576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8a8d8-d6ec-41c8-bc86-189c0a5c46fd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.814815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8a8d8-d6ec-41c8-bc86-189c0a5c46fd None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.816547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 197/403] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:44:59.821683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:44:59.821959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:44:59.822228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:44:59.841521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:44:59.842732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:44:59.888532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 207/404] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 1 17:44:59 2022] PUT /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:44:59.891946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:44:59.930825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 198/405] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:45:00.031693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.072649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:00.143352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d523a7-5648-4855-9c35-dd0799076d6f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.178082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.178718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.178995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.179266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.189834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:00.195480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:00.196394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:00.197213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 208/406] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:00.208645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.298113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.341023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfc5e5-ac3e-4701-bf96-39a3f44fc562 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.378331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.379513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.379791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.380251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.380949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:00.381251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:00.382613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:00.396427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 199/407] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:00.397900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.442426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:00.458355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:00.463545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269356d5-14d6-4fc3-94d9-9ed98d11a3cc None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.498883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e598b-0a5b-42e8-9a03-4e6bb75d5559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.499520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e598b-0a5b-42e8-9a03-4e6bb75d5559 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.499805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e598b-0a5b-42e8-9a03-4e6bb75d5559 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.500080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e598b-0a5b-42e8-9a03-4e6bb75d5559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.507141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec499e-dc9c-418e-a899-360e35889eaf None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.512397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e598b-0a5b-42e8-9a03-4e6bb75d5559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-db7e598b-0a5b-42e8-9a03-4e6bb75d5559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:00.517758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:00.518643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:00.519889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 209/408] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:00.520780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160478-63f2-4d16-8519-008735f5db88 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.520780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160478-63f2-4d16-8519-008735f5db88 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.521075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160478-63f2-4d16-8519-008735f5db88 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.522251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 210/409] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:00.527113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160478-63f2-4d16-8519-008735f5db88 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.547787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a439814-e104-440f-a082-48a5ac9ec490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.548812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439814-e104-440f-a082-48a5ac9ec490 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.549285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439814-e104-440f-a082-48a5ac9ec490 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.549856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439814-e104-440f-a082-48a5ac9ec490 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.551797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439814-e104-440f-a082-48a5ac9ec490 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:00.553477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439814-e104-440f-a082-48a5ac9ec490 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:00.560224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 200/410] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:00.561794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.562342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.562893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.581644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:00.583370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:00.638276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160478-63f2-4d16-8519-008735f5db88 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.655539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 201/411] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:00.656797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.695004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.697423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.697901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.698398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.699361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:00.700593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:00.705116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:00.727865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:00.730804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 211/412] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:00.735998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.783419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:00.794914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45c1c0-895f-4eb0-ae5b-16b2cde43be1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.860243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.860822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.861103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.861378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.863920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:00.865381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:00.867037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 202/413] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:00.873296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.932774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca7b1-0fc7-40a2-bc59-3242ca4f6a82 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.968467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:00.969436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:00.970433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:00.971010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] PATH_INFO: `/v3/groups/8747358a89d442a0b2a8df8ad7d4da2c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:00.973514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] RBAC: Authorizing `identity:get_group(group_id=8747358a89d442a0b2a8df8ad7d4da2c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:00.975851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:00.978871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 212/414] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/groups/8747358a89d442a0b2a8df8ad7d4da2c => generated 229 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:00.984524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:00.990726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef8708-9deb-44ab-9598-fe83c4f43433 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:00.998739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.027981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.029121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.029596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.030180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.032741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] RBAC: Authorizing `identity:create_grant(project_id=cc7140fe2ff44d7a9a15836c34f27dfc, group_id=1682db9ef7bd421aa7465a458e0f8b26, role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.036350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.070810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.086325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54360f72-d97d-4d61-8675-797d087d7476 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.111514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 203/415] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 1 17:45:00 2022] PUT /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc/groups/1682db9ef7bd421aa7465a458e0f8b26/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:01.112896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.121819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.122532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.122854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.125927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.135264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:01.141364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:01.142343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:01.143219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 213/416] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:01.151825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.275794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3fe1b-856f-4bce-b51b-96b1f51be89a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.313549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ddd95-f89b-49aa-b8a4-33cf634b271d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.315488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.316228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.316710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.317017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.317707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:01.318015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.319497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc53ab-2e96-4d0b-aac3-98c6de8636e5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.329286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 214/417] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:01.330573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64108bc-9c24-46b9-9851-40d48510cfa0 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.330859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64108bc-9c24-46b9-9851-40d48510cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.331136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64108bc-9c24-46b9-9851-40d48510cfa0 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.333315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 215/418] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:01.334901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64108bc-9c24-46b9-9851-40d48510cfa0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.347983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.365009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255cd74-7888-40fb-ac4a-dbb33589ac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.365009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255cd74-7888-40fb-ac4a-dbb33589ac29 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.365009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255cd74-7888-40fb-ac4a-dbb33589ac29 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.365009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255cd74-7888-40fb-ac4a-dbb33589ac29 None admin] PATH_INFO: `/v3/projects/195190be58ff49f7a586c1baca5891d6` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.366887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255cd74-7888-40fb-ac4a-dbb33589ac29 None admin] RBAC: Authorizing `identity:get_project(project_id=195190be58ff49f7a586c1baca5891d6)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.368960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255cd74-7888-40fb-ac4a-dbb33589ac29 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.372220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 204/419] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/projects/195190be58ff49f7a586c1baca5891d6 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:01.372220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.372220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.372220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.391331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:01.392547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:01.425686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.462075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64108bc-9c24-46b9-9851-40d48510cfa0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.481789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 205/420] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:01.483290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.503237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.504578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.504802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.505075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.585865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.587447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.598064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.618139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ba53f-f2b4-426f-af96-8af91c84bcd7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.657557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.658563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.659065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.659543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] PATH_INFO: `/v3/projects/195190be58ff49f7a586c1baca5891d6/groups/8747358a89d442a0b2a8df8ad7d4da2c/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.661760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] RBAC: Authorizing `identity:create_grant(project_id=195190be58ff49f7a586c1baca5891d6, group_id=8747358a89d442a0b2a8df8ad7d4da2c, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.663580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.724393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 216/421] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:45:01.730030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.752617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 206/422] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 1 17:45:01 2022] PUT /identity/v3/projects/195190be58ff49f7a586c1baca5891d6/groups/8747358a89d442a0b2a8df8ad7d4da2c/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:01.850245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:01.856075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4177a7-4a48-4f6f-916a-7f4ac15e3a16 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:01.872147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.891816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:01.924906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:01.926160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:01.926439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:01.926710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:01.928680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:01.930158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:01.931879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 217/423] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:01.939386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.019239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8a737-6a55-42ba-af83-acdaf808394d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.057560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.058200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.058481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.058754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.062581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ca685-2126-4cf2-a402-46e13062035f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.069782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:02.074979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:02.075884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:02.076744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 207/424] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:02.082979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.105231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.105870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.106182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.106460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.107590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.109040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.110900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 218/425] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:01 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:02.117104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.165497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.175193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.192819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7f0a-0d7b-45f1-9a31-f47fa09410d3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.221949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ca15f-e26c-4518-90fb-3f0c7058bc02 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.231907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.232552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.232832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.233103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.233781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:02.234213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.236337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.251731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 208/426] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:02.263059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.265763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.266646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.266922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.267191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.267922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=7de433353c124372b395806b9f1eb576) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:02.268225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.270572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24400b-f3d3-421b-b7bf-bb71d1e361d3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.277354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 219/427] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:02.285040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a0a26-fae9-4930-9c40-afc6d391797b None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.285492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a0a26-fae9-4930-9c40-afc6d391797b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.285791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a0a26-fae9-4930-9c40-afc6d391797b None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.287573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 220/428] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:02.293853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.294147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.294396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.326334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:02.330859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:02.446815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a931d-6b8f-4e39-a7fa-4bc374962711 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.470568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 221/429] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:02.492046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.493041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.493464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.493878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.517200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.591301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.593122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.736347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 209/430] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:45:02.745632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.799011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.811773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.867845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:02.901488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f5722-2e97-46cb-850c-b6331302e4f3 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.948088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775e8f3-aadc-443f-aa20-439bc564414d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:02.969274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.971198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.971754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.972787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] PATH_INFO: `/v3/projects/7de433353c124372b395806b9f1eb576` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:02.975101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] RBAC: Authorizing `identity:get_project(project_id=7de433353c124372b395806b9f1eb576)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:02.977172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:02.979492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 222/431] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/projects/7de433353c124372b395806b9f1eb576 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:02.984692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:02.995833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:02.996783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:02.997253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:02.997701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.014658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:03.020537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:03.021691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 17:45:03.023334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:03.026537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 210/432] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:03.027913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.109884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7570d6-f323-45dd-a9fa-b9c9d91fe7ab None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.119209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.151209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.152423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.152922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.153404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.154942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.156992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.159087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 223/433] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:02 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:03.164548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.168279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3ecf2-2b86-4397-a80e-875e7f9e6af2 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.181322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.214798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.215279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.215561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.215851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.216738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] RBAC: Adding query filter params (name=member) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:03.217045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.218507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.236057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 211/434] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:03.237521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.321471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135756e-5233-445f-a0fc-f7bb63d213d8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.379161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.379804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.380087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.380391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.381226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=7de433353c124372b395806b9f1eb576) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:03.381533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.383055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437443a-7453-4de9-96bb-fa197c45fbc1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.390699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 224/435] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=7de433353c124372b395806b9f1eb576&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:03.409150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ddbec-c4c2-4a04-be76-bfd0a67bf272 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.456914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.457839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.457995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.458214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.511006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.558761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.559828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a350d-98c3-4080-b546-49da956f7392 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.560118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a350d-98c3-4080-b546-49da956f7392 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.560280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.560504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a350d-98c3-4080-b546-49da956f7392 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.562105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 225/436] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:03.569302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.569603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.569853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.592172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:03.593414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:03.655290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.711232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 226/437] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:03.714270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 212/438] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:03.720454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.827631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c52e4-d8f1-4aa4-907c-c32f498bc8df None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:03.864108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:03.864786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:03.865061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:03.865329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:03.866492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:03.867953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:03.869945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 227/439] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:03.875630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.949707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:03.980974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:03.984192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27e25-7734-4412-bad1-26f2dfe0c61f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.018928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.023076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.023714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.023994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.024264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.025863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.027341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.029449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 213/440] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:04.035414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.089929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d4eff-2672-4ea5-af2e-a9e38c8f82d9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.126169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.126796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.127072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.127345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.138547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:04.142771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cdea6-0bf7-4cb2-a8e5-3f1f046288a7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:04.143932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:04.144862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:04.145661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 228/441] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:04.150913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.181259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.181863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.182192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.182468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.183202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=6bc7a311754c40b2bbf21dccc998bc9b, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:04.183511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.184987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e82d0-08ad-496b-979c-e847713316f5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.187835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.195412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 214/442] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=6bc7a311754c40b2bbf21dccc998bc9b => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:04.260637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.264758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835672d8-4430-4bef-af4d-aa2511a0946a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.265048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835672d8-4430-4bef-af4d-aa2511a0946a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.265319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835672d8-4430-4bef-af4d-aa2511a0946a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.267049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 215/443] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:04.267740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d38abb-3c54-4ad6-93af-3c6c5290a4da None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.272773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.274059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.274059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.293027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:04.294898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:04.307878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.308523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.308797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.310525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.310525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:04.310525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.311530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.324255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 229/444] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:04.330209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.377642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 216/445] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:45:04.447431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa07326-fef0-4c21-ae1c-fe422b17614c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.482807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.483419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.483700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.483971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.514083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.559116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.560611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.653485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.657070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 230/446] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:45:04.663325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.721965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.753945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e04e4f-b0cd-4ecb-9c10-3afacda51de2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.763373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf8401-500f-42f6-83e5-d8194fbe9eb0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.786975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.792487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.793120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.793400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.793858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.803850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.804225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:04.804661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.804945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.805217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] PATH_INFO: `/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.806319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] RBAC: Authorizing `identity:get_project(project_id=cc7140fe2ff44d7a9a15836c34f27dfc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.807734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:04.809176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:04.810089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:04.810916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:04.811621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 231/447] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/projects/cc7140fe2ff44d7a9a15836c34f27dfc => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:04.811691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 217/448] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:04.814923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.816112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.913949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503cff5-d1ae-4571-b253-d29d95477989 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.916089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d35f6-4a0b-4482-b201-09d6898c5f4f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:04.941202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:04.949418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.954584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:04.954702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.954777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:04.954845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.954927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:04.954997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] PATH_INFO: `/v3/groups/1682db9ef7bd421aa7465a458e0f8b26` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.955063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:04.955063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:04.955063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.955239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] RBAC: Authorizing `identity:get_group(group_id=1682db9ef7bd421aa7465a458e0f8b26)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:04.955342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.955411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:04.955411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 232/449] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/groups/1682db9ef7bd421aa7465a458e0f8b26 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:04.961230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:04.962060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 218/450] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:04.967284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.023196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.061705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52937-4556-4a76-81ca-50ebb6a2f1ad None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.069884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca4ecd-7ad9-47a3-a9d0-564fde415a4d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.094762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.095360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.095635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.095912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.096655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] RBAC: Adding query filter params (group.id=1682db9ef7bd421aa7465a458e0f8b26, role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:05.096959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.098998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.103781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.104398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.104675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.104783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 233/451] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/role_assignments?group.id=1682db9ef7bd421aa7465a458e0f8b26&scope.project.id=cc7140fe2ff44d7a9a15836c34f27dfc&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:05.104986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.177084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.178472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.232182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.283031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 219/452] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 17:45:05.289156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.349944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.392919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90668da-21d2-4d3f-8a0a-79a5c39c35cc None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.426825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.427521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.427746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.428021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] PATH_INFO: `/v3/projects/195190be58ff49f7a586c1baca5891d6` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.429186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] RBAC: Authorizing `identity:get_project(project_id=195190be58ff49f7a586c1baca5891d6)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.430611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.432242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 234/453] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/projects/195190be58ff49f7a586c1baca5891d6 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:05.437817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.537024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb753fa-16c6-4976-9f46-1f041e82a11e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.569975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960716b-23eb-4500-9d99-0b434376c04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.570583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960716b-23eb-4500-9d99-0b434376c04a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.570860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960716b-23eb-4500-9d99-0b434376c04a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.571128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960716b-23eb-4500-9d99-0b434376c04a None admin] PATH_INFO: `/v3/groups/8747358a89d442a0b2a8df8ad7d4da2c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.572210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960716b-23eb-4500-9d99-0b434376c04a None admin] RBAC: Authorizing `identity:get_group(group_id=8747358a89d442a0b2a8df8ad7d4da2c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.573655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960716b-23eb-4500-9d99-0b434376c04a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.575189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 220/454] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/groups/8747358a89d442a0b2a8df8ad7d4da2c => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:05.581064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:05.677219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b9367-b3b4-47fe-9c26-a6b227c2059b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:05.701580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:05.709800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:05.710394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:05.710674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:05.710947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:05.711674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] RBAC: Adding query filter params (group.id=8747358a89d442a0b2a8df8ad7d4da2c, role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=195190be58ff49f7a586c1baca5891d6) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:05.711984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:05.713429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13cd8b-6612-4420-9406-f6bcd442d571 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:05.718907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 235/455] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 1 17:45:05 2022] GET /identity/v3/role_assignments?group.id=8747358a89d442a0b2a8df8ad7d4da2c&scope.project.id=195190be58ff49f7a586c1baca5891d6&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:07.942445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c352b8a-4cbd-4d3a-85c9-ab426bf7b03d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:07.943458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c352b8a-4cbd-4d3a-85c9-ab426bf7b03d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:07.943960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c352b8a-4cbd-4d3a-85c9-ab426bf7b03d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:07.945864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 221/456] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:07.950761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73119b-c755-44ee-88a0-e14e8b0749f5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:07.951160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73119b-c755-44ee-88a0-e14e8b0749f5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:07.951329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73119b-c755-44ee-88a0-e14e8b0749f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:07.981791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-6b73119b-c755-44ee-88a0-e14e8b0749f5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:07.982975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6b73119b-c755-44ee-88a0-e14e8b0749f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:08.070865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 236/457] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:08.154973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170cf9-5f23-4497-97fd-877d3af2c0b6 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.155296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170cf9-5f23-4497-97fd-877d3af2c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.155516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170cf9-5f23-4497-97fd-877d3af2c0b6 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.157070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 222/458] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:08.162567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc353a-3503-42eb-87f7-1fd6b8bbcbef None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.162855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc353a-3503-42eb-87f7-1fd6b8bbcbef None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.163133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc353a-3503-42eb-87f7-1fd6b8bbcbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.189309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-cfdc353a-3503-42eb-87f7-1fd6b8bbcbef None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:08.190496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cfdc353a-3503-42eb-87f7-1fd6b8bbcbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:08.286036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 237/459] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:08.352176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fa9e3-34bd-4a08-9c4e-e4b00dee4c89 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.352534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fa9e3-34bd-4a08-9c4e-e4b00dee4c89 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.352754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fa9e3-34bd-4a08-9c4e-e4b00dee4c89 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.354376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 223/460] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:08.359401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.359689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.359957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.385333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:08.386520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:08.454369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:08.485964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 238/461] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:08.527854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fa9e3-34bd-4a08-9c4e-e4b00dee4c89 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:08.555024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:08.658353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fa9e3-34bd-4a08-9c4e-e4b00dee4c89 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:08.703676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:08.704298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.704597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.704863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.715522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:08.717011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:08.735860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:08.736861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:08.738348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:08.738348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:08.738348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:08.741399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 224/462] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:08.747253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:08.757773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:08.852848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:08.885192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25c384-9a46-4dde-a676-9488cad71cc0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:08.886661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416cb6-ee49-48f8-a5f3-7d52ba1e0508 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:08.924680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:08.926072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.926349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.926680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.927770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:08.928417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:08.928696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:08.929211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:08.930286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:08.930604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:08.931983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:08.938945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:08.940625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:08.945905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 239/463] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:08.946754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:08.956185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:08.959135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:08.961421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:08.961421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:08.961421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:08.961421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 225/464] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:08.961421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:08.994672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:09.055156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44e9b-923a-4179-b4ae-822359da5da9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.073297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0b916-e9a7-4096-a065-44e0db1e2681 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.089395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.090031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.090298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.090583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.099434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:09.101625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:09.103161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:09.110751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928f04e-0511-4b97-85ca-953838f530b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.111753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928f04e-0511-4b97-85ca-953838f530b3 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.113505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928f04e-0511-4b97-85ca-953838f530b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.113505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928f04e-0511-4b97-85ca-953838f530b3 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.113809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928f04e-0511-4b97-85ca-953838f530b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:09.115296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928f04e-0511-4b97-85ca-953838f530b3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:09.118816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:09.119733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:09.120962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:09.120962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:09.120962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:09.124912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 240/465] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:09.126261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:09.217759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:09.218164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 226/466] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:45:08 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:09.219465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e350db7-a758-420d-91b5-0c886614b2be None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.219745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e350db7-a758-420d-91b5-0c886614b2be None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.220059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e350db7-a758-420d-91b5-0c886614b2be None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.226489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 227/467] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:09.226489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e350db7-a758-420d-91b5-0c886614b2be None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:09.243318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7ee5-f578-4c88-ae67-b26e5502538d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.297735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.299394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.299900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.300996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.302268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:09.302845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:09.305390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28854696-ad17-4ddf-8c61-f287f839de04 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:09.322773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 241/468] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:09.324506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.324995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.325577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.354024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:09.356371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:09.389528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e350db7-a758-420d-91b5-0c886614b2be None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.437471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.438735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.439714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.439714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.439996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:09.440264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:09.441716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:09.442857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 242/469] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:09.443818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:09.461752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 228/470] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:09.467668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:09.518089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:09.540846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:09.553947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38001c-06c8-4fd0-b0bf-75b9cf48a43c None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.589328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.590118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.590406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.590681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.591389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:09.592979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:09.618703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0987e-d32d-4c89-ba26-afd6b80c674e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.654507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04869f6d-27c0-4466-b943-6d2081b7f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.655159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04869f6d-27c0-4466-b943-6d2081b7f4dd None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.655442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04869f6d-27c0-4466-b943-6d2081b7f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.655713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04869f6d-27c0-4466-b943-6d2081b7f4dd None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.656384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04869f6d-27c0-4466-b943-6d2081b7f4dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:09.657942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04869f6d-27c0-4466-b943-6d2081b7f4dd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:09.724440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 243/471] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:45:09 2022] POST /identity/v3/users => generated 248 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:09.751784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 229/472] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:45:09 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:09.776912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:09.834667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f331c-5e96-4dd6-830c-ea8bc9df5fe9 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.834988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f331c-5e96-4dd6-830c-ea8bc9df5fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.835217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f331c-5e96-4dd6-830c-ea8bc9df5fe9 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.837054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 230/473] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:09.854150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.854477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.855228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.889465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:09.891033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:09.937318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09594c-e210-4bca-8310-a849477ced62 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:09.986544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:09.987554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:09.989186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:09.989461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:09.993986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 231/474] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:09.998238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:10.003776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:10.005312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:10.023270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:10.024462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:10.025337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:10.025337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:10.025337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:10.025337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 244/475] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:10.030487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:10.059812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:10.148130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941c7fd-4fe9-47ac-84ab-aaac8870b166 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:10.183915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:10.184563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:10.184844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:10.185116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:10.185768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:10.186070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:10.187468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:10.199654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 232/476] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:10.204879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:10.291662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:10.363019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d91533-9b75-403e-8580-87f289bd281b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:10.400513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:10.401048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:10.401324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:10.401590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:10.402238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:10.404204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:10.460733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:10.468910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff2d7-57c2-4e9c-a917-0c845cbc489e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:10.504469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:10.505079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:10.505396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:10.505671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:10.506398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:10.507870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:10.522476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 245/477] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:45:10 2022] POST /identity/v3/users => generated 249 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:10.549388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.sql.core [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:45:10.549388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 17:45:10.549388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [parameters: {'id': '2100915b8fae4f6882cdecd9a1c8fed5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 17:45:10.549388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81703) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result.read() Sep 01 17:45:10.570290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:45:10.571563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application yield resource Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise self.value Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application session.commit() Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:45:10.572839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:45:10.573723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:45:10.574612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result.read() Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:45:10.575426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [parameters: {'id': '2100915b8fae4f6882cdecd9a1c8fed5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:10.576246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 17:45:10.579186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:10.581220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 233/478] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 1 17:45:10 2022] POST /identity/v3/roles => generated 144 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:10.588139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:10.652804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:10.711566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342cb7d-0533-4bce-8af6-9eabc00758fb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:10.746387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c5eec-1ee2-4375-b17b-96a69306e09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:10.747039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c5eec-1ee2-4375-b17b-96a69306e09a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:10.747322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c5eec-1ee2-4375-b17b-96a69306e09a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:10.747590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c5eec-1ee2-4375-b17b-96a69306e09a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:10.759769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c5eec-1ee2-4375-b17b-96a69306e09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f82c5eec-1ee2-4375-b17b-96a69306e09a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:10.766041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:10.768290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:10.769869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 246/479] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:10.777146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:10.886947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee9d32-17d3-4ca5-9008-850709942a4f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:10.922205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:10.922850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:10.923229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:10.923585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:10.924341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:10.924869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:10.927063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4009-d96d-461a-8ab9-b30f21bb633a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:10.935996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 234/480] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:11.703932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753797f-f116-4060-8870-a6e697ad4e7e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.704502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753797f-f116-4060-8870-a6e697ad4e7e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.704502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753797f-f116-4060-8870-a6e697ad4e7e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.706075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 247/481] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:11.711504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf1e45-6a6b-4871-8d51-37725343e0a7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:11.711913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf1e45-6a6b-4871-8d51-37725343e0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:11.712087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf1e45-6a6b-4871-8d51-37725343e0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:11.737113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6cbf1e45-6a6b-4871-8d51-37725343e0a7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:11.738495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6cbf1e45-6a6b-4871-8d51-37725343e0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:11.746761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:11.827377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 235/482] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:12.123508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2628dfd-60cf-43df-b67c-b9607a012de4 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.123867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2628dfd-60cf-43df-b67c-b9607a012de4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.124050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2628dfd-60cf-43df-b67c-b9607a012de4 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.125700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 248/483] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:12.131523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b0d35-4cde-4960-8d8c-2fc1d71bc452 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.131816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b0d35-4cde-4960-8d8c-2fc1d71bc452 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.132092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b0d35-4cde-4960-8d8c-2fc1d71bc452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.160068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-615b0d35-4cde-4960-8d8c-2fc1d71bc452 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:12.161740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-615b0d35-4cde-4960-8d8c-2fc1d71bc452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:12.264952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 236/484] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:12.282265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2628dfd-60cf-43df-b67c-b9607a012de4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.340657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bd83-aeef-4d88-abc8-95d4e9f5c036 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.341306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bd83-aeef-4d88-abc8-95d4e9f5c036 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.341772 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bd83-aeef-4d88-abc8-95d4e9f5c036 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.343541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 237/485] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:12.349337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.349623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.349902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.370657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:12.372676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:12.459794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 238/486] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:45:12.466096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2628dfd-60cf-43df-b67c-b9607a012de4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.480296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:12.525900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.526536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.526812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.527088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.542691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:12.547646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:12.548630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:12.549878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 249/487] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:12.555330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.559192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:12.601617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.694756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb521fc0-6d8b-45ae-9edd-76fc730ed92b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.756739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.757358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.757580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.757804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.758467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:12.758764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:12.760139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:12.774488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 239/488] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:12.780593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.809868 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5456ebf-445b-4757-bf8e-52972baca90c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.887442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.890255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:12.890916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.893315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.894147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.912036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:12.918750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:12.919649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:12.920473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:12.922522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 250/489] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:12.923089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:12.937663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d97a2-a7c0-4bd9-af04-d4c683c510eb None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:12.961224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:12.974893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:12.977063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:12.977063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:12.977063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:12.988605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:12.990300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:13.009174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:13.010144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:13.011539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:13.011539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:13.011539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:13.015092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 240/490] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:13.016108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.066657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb31dc-bf82-4b7f-8dbc-a79c8cd6d902 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.112967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.113971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.114581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.118525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.131044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f64e5e-9e4c-4f13-9d5b-a60273541da1 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.132217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:13.137455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:13.138351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:13.139371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 251/491] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:13.140729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.156047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.181689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.182290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.182564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.182892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.183559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:13.183855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.185353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.195845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 241/492] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:13.196813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.246445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.265677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b160ac-d287-44c8-bc89-649a5b96108b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.303274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.303886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.304163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.304472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.305133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:13.305434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.306855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.307144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19868c7-0e0f-41b4-8a4f-512594395edc None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.318804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 252/493] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:13.319644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.345869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.346471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.346748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.347083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.347746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:13.348050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.349448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428e62e-82bf-4766-b586-4e08732fe95a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.357159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 242/494] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:13.358298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5a229-ac7f-41a5-aff0-bf07f7043d29 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.358577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5a229-ac7f-41a5-aff0-bf07f7043d29 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.358845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5a229-ac7f-41a5-aff0-bf07f7043d29 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.360038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 243/495] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:13.361055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4210-5700-43b1-91ba-defa3aba995d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.361334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4210-5700-43b1-91ba-defa3aba995d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.361884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4210-5700-43b1-91ba-defa3aba995d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.363627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 244/496] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:13.365309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4210-5700-43b1-91ba-defa3aba995d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.409071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.422837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2725356-0a36-476a-8734-f881f66634f6 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.457419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.458040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.458317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.458614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.467840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4210-5700-43b1-91ba-defa3aba995d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.468817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.470250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.477314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:13.485102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:13.487138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:13.488743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:13.488743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:13.488743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:13.491735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 253/497] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:13.492608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.503776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.504437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.504718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.504991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.512579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.514020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-2030ffa8-057b-41dd-ae73-7f1c2dfbc2e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:13.526911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:13.527982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:13.528952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 245/498] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:13.529546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9454b15-8003-4374-a877-cebb138fbb5e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.529546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9454b15-8003-4374-a877-cebb138fbb5e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.529695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9454b15-8003-4374-a877-cebb138fbb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.548140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d9454b15-8003-4374-a877-cebb138fbb5e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:13.549378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d9454b15-8003-4374-a877-cebb138fbb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.594245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab24581-6a08-49e2-b3ee-6389fd6bb838 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.616035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 246/499] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:13.617362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.617659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.617935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.627983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.628649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.628956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.629227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.640202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.641922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.643191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:13.644642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:13.656895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:13.657810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:13.658680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:13.658680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:13.658680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:13.658680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 254/500] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:13.659972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.671198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.714035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 247/501] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:13.721185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.725995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.792747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47489c2-62ca-4512-8d4b-caed7fbd8b6b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.835215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.835970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.836243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.836552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.837204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:13.837514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.839282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.855385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 255/502] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:13.856300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.856634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6a6c8-274b-4f9c-af63-6407ab0bd2cd None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.900804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:13.901688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:13.902123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:13.902546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:13.903427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:13.904258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:13.906236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:13.948903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 248/503] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:13.949876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:13.965415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51ae5c-8a19-4a10-9a28-d8e0c44eaf91 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:13.971656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.976694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:13.999515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.000157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.000466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.000741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.001381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:14.001683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.003035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.013941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 256/504] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:14.014738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.055194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025cd1-7982-449a-a2c5-eda8bfbdf06b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.088555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.089181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.089455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.089722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.096713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.098804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:14.110442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:14.111330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:14.112223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 249/505] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:14.112906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.116070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587ea81-ce90-4b35-81bb-3c6b74d425e0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.153505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.154136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.154418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.154694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.165522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:14.170765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:14.171710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:14.172568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 257/506] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:14.173248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.195823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.203804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.217668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8efa6-00ff-4741-9bb6-d6ed521709e7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.251713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.251713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.252029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.252194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.273931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905d9e5-6839-46a4-b300-462d7709ce36 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.306467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.307024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.307299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.307566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.317972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.319398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.323685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.325166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:14.336010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:14.336986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:14.337854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:14.337854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:14.337854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:14.337854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 258/507] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:14.338080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.409617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.417529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.431035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 250/508] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:14.431862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.439419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2652-19b6-4479-9250-3734e68dfca0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.472717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.473331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.473606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.473873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.481653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.483073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:14.495250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:14.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 259/509] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:14.497297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.534012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fcffd-5ce5-438f-8c9a-e0ae443749cc None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.568008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.568571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.568852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.569119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.569840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:14.570140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.571577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.589099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 251/510] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:14.589885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.603908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200fff6-67c6-4e16-addd-1377d833eb5f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.623175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.638845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.639439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.639717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.639987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.640694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:14.640993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.642603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.650477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 260/511] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:14.651264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.655271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.687963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d22b1-ed31-4473-b28b-605369a13b58 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.722233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.722826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.723103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.723371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.724010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:14.724334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.725694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.736700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 252/512] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:14.737482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.754557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8c543-8a17-4638-927e-5740bb66bbcc None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.791457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.792085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.792396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.792682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.803593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.805030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:14.819874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:14.820794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:14.821662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:14.821662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:14.821662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:14.821662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 261/513] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:14.823362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.835687 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.841589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c36a30-ce6d-4f3b-a40c-5ba66b1b003e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.874727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.875387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.875619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.875889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.876644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:14.876943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:14.877649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:14.879020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:14.897382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 253/514] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:14.898194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:14.927028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77ab55-a5ba-420b-97da-4d67edb5a562 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:14.960964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:14.961573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:14.961850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:14.962126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:14.996853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c7efa-eebc-41df-b857-49dfd9ae6e69 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.028574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.030019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.030867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.031486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.031766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.032030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.041813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.043224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:15.057397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:15.058375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:15.059226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:15.059226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:15.059226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:15.059226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 254/515] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:15.059226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.068681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.090355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.133311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 262/516] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:15.134206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.163279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8afc5e-f672-481c-8765-a69f8d75b0c1 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.196495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.197098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.197379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.197651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.198292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.199730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.232799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429fc62-9885-4298-aa98-35f199f067c8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.266698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.267288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.267566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.267835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.268500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:15.268882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.270947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.277683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 255/517] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:45:15 2022] POST /identity/v3/users => generated 247 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:15.286967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.287079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 263/518] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:15.287079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.310283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.361249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.406703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e59092-0300-46cb-8dfd-fb81e3822c12 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.412670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7255b-f473-4078-bdb4-5c8bd22168bd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.448141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.448884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.449242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.449523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.450173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:15.450478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.451848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.452908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.453535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.453807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.454400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.466275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 264/519] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:15.467149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.527961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.529455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.551021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.574050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4c406-8cd9-4a9d-ad68-e92df70c73d5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.594110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.610397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.611032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.611366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.611599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.622697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.624134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:15.641671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:15.642625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:15.643484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:15.643484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:15.643484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:15.643484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 265/520] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:15.643484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.644619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 256/521] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:15.646104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.748602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edd8e4-f218-4383-a983-42616b341ab3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.751739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7df8f-28ad-4abc-a417-2136a16bba4b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.781531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.785467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.786175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.786511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.786775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.790614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.791238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.796500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.796500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.796669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.796669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.798491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.799885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:15.807862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:15.809064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:15.809966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 257/522] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:15.811728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:15.812873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:15.813734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:15.814670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:15.818281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 266/523] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:15.818281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.840075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:15.922233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dc5ed-c1cc-4d59-b2a4-48703a5df989 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.929382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32fb56-f720-4990-b0d3-6ddd5300e8ac None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:15.958261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.958889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.959222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.959494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.965403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:15.966008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:15.966286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:15.966555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:15.967187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:15.967482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.968865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.972596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:15.974031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:15.981121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 267/524] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:15.985706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:15.995218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:15.996136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:15.997127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:15.997127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:15.997127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:15.997127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 258/525] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:15.997810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.048551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.075287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.101278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5a06-da65-45ff-afff-4875a708b6c0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.101717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04af8cc-1352-4671-8b79-d3d7758057dd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.136102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.136724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.136999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.137123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.137311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.137617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.137895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.137998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:16.138161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.138258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.138842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:16.139139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.139657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.140567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.165335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 259/526] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:16.165335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.183848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 268/527] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 1 17:45:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:16.183848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.287363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e6208-a807-4432-bab1-78117f7bf764 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.301212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b8f77-4914-4213-92fb-2180ce960413 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.309502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.322695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.323318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.323597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.323866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.325765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.330728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.332820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.338875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.339692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.340151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.340623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.341463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:16.341942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.343469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:16.344658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:16.345631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 260/528] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:16.346500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.358754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 269/529] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:16.363274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.462173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46a291-d38c-4555-8d75-418c1b83e53e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.476163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33fe55-8da8-4335-a5af-625f0a9e0d4f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.497800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.498730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.499149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.499614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.500507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=99dec0395ef74fd5a6023ae06a1b9538) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:16.500953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.502751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.510161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 261/530] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/role_assignments?user.id=99dec0395ef74fd5a6023ae06a1b9538&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:16.510324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.510932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.511212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.511514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.511617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.569965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.589484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.620745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.622542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.646362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a478-c652-4ace-847f-d8cc1fe5c115 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.691401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.692043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.692336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.692784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.693487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:16.693788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.695217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.708785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 262/531] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:16.709904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.757377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 270/532] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:16.758554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.834287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d8afd-6f28-419e-a8ee-48ed394a98fe None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.852962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.854273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:16.872059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.872474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.872753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.873026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.880649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.882100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:16.895080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:16.901465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 263/533] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:16.902329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:16.903730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f551-571b-417f-bf03-18cc2a79af3d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:16.945618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:16.946277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:16.946548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:16.946818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:16.947620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=9c32115f24264e738c432f700a2f7dcc) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:16.947928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:16.949400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:16.960654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 271/534] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/role_assignments?user.id=9c32115f24264e738c432f700a2f7dcc&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:16.961722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:17.039756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e86470-c6ea-41a7-90c7-b517f41029a8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.081602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.082396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.082827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.083175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.085384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb014ea9-d4ca-4957-8e95-b47e7c3b543f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.095613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.097517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:17.115457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:17.116645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:17.117499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:17.117499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:17.117499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:17.117499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 264/535] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:17.123540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:17.132329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:17.146550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.147178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.147458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.152497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.152497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:17.152497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.152497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.187166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:17.192572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 272/536] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:17.198994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:17.271611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424298d-0385-491a-91cf-ebb9cc3f88ec None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.320674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.321878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.322374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.323065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.324411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:17.325179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.327405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad725c-5e90-4c59-bf0e-b42707b6616a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.343845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 265/537] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:17.348663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24443-9701-4932-8242-98fac87f5ba5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.348790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a78009-b366-49cc-a257-f2f4bb99c912 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.349056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a78009-b366-49cc-a257-f2f4bb99c912 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.349817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a78009-b366-49cc-a257-f2f4bb99c912 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.352140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 266/538] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:17.356816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a78009-b366-49cc-a257-f2f4bb99c912 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:17.396091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.397094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.397599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.398082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.399008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=dbb0ec082bdb45b0bb2c59cbad440d99) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:17.399511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.401686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b1465-50e1-4a03-a71c-ceb1ebf32298 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.416638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 273/539] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:17 2022] GET /identity/v3/role_assignments?user.id=dbb0ec082bdb45b0bb2c59cbad440d99&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:17.418634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.419123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.419644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.457632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:17.458212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:17.460106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:17.472342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a78009-b366-49cc-a257-f2f4bb99c912 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.510910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.511497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.511773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.512043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.524344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.524344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:17.545732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:17.546659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:17.547503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 267/540] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:17.553903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:17.563568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:17.603386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 274/541] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:17.686942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f535e02-41d9-455e-9a64-c419e8ede8eb None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.723945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.724576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.724852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.725125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.725818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:17.726118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.727544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.744532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 268/542] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:17.746987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:17.878699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa2c0-3334-4323-a94f-9cf830bb2633 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:17.929663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:17.930280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:17.930557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:17.930830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:17.931554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=c16d573ee440426aaa09eddb9c2658dc) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:17.931917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:17.933419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:17.940848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 275/543] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:17 2022] GET /identity/v3/role_assignments?user.id=c16d573ee440426aaa09eddb9c2658dc&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:18.002655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:18.016081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:18.133223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599f854-d64f-4ad5-93d8-0737b302307e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:18.172415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:18.173050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:18.173324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:18.173591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:18.185151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:18.190957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:18.191996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:18.192843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 269/544] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:18.200547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:18.282476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:18.372041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd5fcc-f2f5-42e5-a470-c67d69e03f37 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:18.407263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:18.408197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:18.408722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:18.409287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:18.410714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:18.411223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:18.412812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:18.421547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 276/545] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:18.428433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:18.545228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:18.594103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc6f6-8f3c-4cb6-8cb6-63f5e84adbcd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:18.642213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:18.642842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:18.643117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:18.643385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:18.657046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:18.658549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-30ccffad-53c2-4d8d-85eb-fccf1ffc4f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:18.675910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:18.676938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:18.677775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:18.677775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:18.677775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:18.680397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 270/546] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:18.682889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:18.809378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398a3dc-f722-4d66-930e-2f21afbc015c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:18.821045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:18.844788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:18.852948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:18.852948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:18.852948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:18.852948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:18.852948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:18.852948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:18.853438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc57c20-49c8-4697-8ce9-06c93de7e96b None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:18.853438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc57c20-49c8-4697-8ce9-06c93de7e96b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:18.853438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc57c20-49c8-4697-8ce9-06c93de7e96b None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:18.853438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 271/547] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:18.861603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:18.861895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 277/548] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:18.862024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:18.862362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:18.868357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:18.895455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:18.896708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:19.022643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 272/549] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:19.039048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b753f89-72e5-4206-8016-50960b74fa1d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.076948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.077874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.078291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.078701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.086476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:19.088081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:19.100183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:19.101332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:19.102483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:19.104597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 278/550] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:19.111719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.196734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:19.244146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d09f7-12ae-4fd7-a3b1-0036bd3f6da7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.283813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.284491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.284973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.285268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.286016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:19.286322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:19.287762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:19.306902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 273/551] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:19.312096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.345690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.361393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:19.454905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1953-48b1-4453-8b64-293190ebbe43 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.461147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b613-6f93-440e-b9d0-67f6fdb70fab None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.470295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:19.499653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b207518-a1fe-4fe3-bd94-1ed2645490ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.500268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b207518-a1fe-4fe3-bd94-1ed2645490ce None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.500572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b207518-a1fe-4fe3-bd94-1ed2645490ce None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.500843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b207518-a1fe-4fe3-bd94-1ed2645490ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.504613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa25e1-7c45-4601-87e7-64f283368a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.506365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa25e1-7c45-4601-87e7-64f283368a9b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.506825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa25e1-7c45-4601-87e7-64f283368a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.507274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa25e1-7c45-4601-87e7-64f283368a9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.520971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa25e1-7c45-4601-87e7-64f283368a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-a7aa25e1-7c45-4601-87e7-64f283368a9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:19.527554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:19.528497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:19.529416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:19.536887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 274/552] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:19.538565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fa864-1b2a-43ab-b812-06dc4e59e8ec None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.538849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fa864-1b2a-43ab-b812-06dc4e59e8ec None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.539118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fa864-1b2a-43ab-b812-06dc4e59e8ec None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.540679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 275/553] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:19.542443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fa864-1b2a-43ab-b812-06dc4e59e8ec None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.596628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b207518-a1fe-4fe3-bd94-1ed2645490ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:19.598638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b207518-a1fe-4fe3-bd94-1ed2645490ce None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:19.636349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:19.681917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fa864-1b2a-43ab-b812-06dc4e59e8ec None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.715517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 279/554] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:19.715517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.715517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.715517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.725364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.726304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.726832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.727300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.728163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:19.728696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:19.730447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:19.732031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:19.733261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:19.745629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 276/555] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:19.746959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.783043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:19.811683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 280/556] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:19.812465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.859946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2354-b4f2-466c-bd4c-21000ef3f9c3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.874982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:19.899570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.900651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.901130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.901598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.913950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:19.915665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:19.920556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6714c7-61f2-4c61-b74b-55469678f14d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:19.931621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:19.932656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:19.933543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:19.933543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:19.933543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:19.935688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 277/557] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:19.941347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:19.958367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:19.959000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:19.959275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:19.959543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:19.978456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:19.979760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:20.011621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:20.013115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:20.013999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:20.013999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:20.013999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:20.014884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 281/558] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:20.023238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.091192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.116085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3c2b7-10cf-41e3-9232-fd74d589f832 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.160845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ef9b-3ac3-4dbc-bf5b-8b9b968f9d3e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.164397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.165077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.165422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.165762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.166481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:20.166850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.168275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9ebc7-98f9-4091-a497-1ff2e15b8ac4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.176022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.186357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 278/559] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:20.188623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631609-4357-4539-bea7-1f297a5d7ee6 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.188976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631609-4357-4539-bea7-1f297a5d7ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.189370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631609-4357-4539-bea7-1f297a5d7ee6 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.196350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 279/560] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:20.196350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16631609-4357-4539-bea7-1f297a5d7ee6 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.203327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.204440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.204903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.205356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.206786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:20.207327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.209027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6b925-8721-4b42-9393-2a563fe7b09b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.221890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 282/561] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:20.223560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.223841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.224111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.245746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:20.246932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:20.302686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16631609-4357-4539-bea7-1f297a5d7ee6 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.317729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 283/562] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:20.318566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.348583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.349133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.349562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.349830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.358096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.359920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:20.373498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:20.374376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:20.375305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 280/563] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:20.375790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.381629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.428697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333d12b-d011-4d3c-bba3-2469616104ca None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.430762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.465433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000b181-cb90-4507-8e28-c0ab0f6f6566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.466071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000b181-cb90-4507-8e28-c0ab0f6f6566 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.466351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000b181-cb90-4507-8e28-c0ab0f6f6566 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.466623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000b181-cb90-4507-8e28-c0ab0f6f6566 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.477609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000b181-cb90-4507-8e28-c0ab0f6f6566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:20.482670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ccf48-6c2c-4f07-bd00-2f962f537eaa None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e000b181-cb90-4507-8e28-c0ab0f6f6566 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:20.482788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:20.483702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:20.484595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 284/564] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:20.485823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a0e99-d193-4443-a24a-c9a4b2fff3cc None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.485823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a0e99-d193-4443-a24a-c9a4b2fff3cc None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.485991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a0e99-d193-4443-a24a-c9a4b2fff3cc None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.487373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 285/565] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:20.487943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a0e99-d193-4443-a24a-c9a4b2fff3cc None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.527752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.528405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.528683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.528950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.536557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.537982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:20.549116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:20.550797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:20.551998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 281/566] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:20.552778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.599854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a0e99-d193-4443-a24a-c9a4b2fff3cc None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.615764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.633361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.633967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.634244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.634517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.635209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:20.635516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.636987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73960f25-fe5e-459d-b6c7-a5aceec02ba6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.649496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 286/567] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:20.650608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.650897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.651201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.657849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7795e0-5bc6-4e3c-ad18-bba9161af499 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.668831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:20.670021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:20.686397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.693828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.699191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.699191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.699191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.699191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:20.699191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.699191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.705277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 282/568] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:20.706091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.740748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 287/569] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:20.741656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.807007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840cc77-0ea0-42a8-a9c4-86cfeb23fdd1 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.840690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.841358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983b47e-42ef-49ee-9872-64801cacb066 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:20.853656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.853656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.853656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.853656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.854011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:20.854011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:20.854011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:20.873258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 283/570] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:20.874369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.882275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:20.882906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:20.883185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:20.883445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:20.894365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:20.899399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:20.900353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 288/571] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:20.901246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:20.916976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:20.990407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e155a3d-e791-4fba-a155-d957aced872a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.006835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa854-aa2b-425f-82c6-3d1e570a107e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.026088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93490-6170-4140-83ba-6918c399044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.026697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93490-6170-4140-83ba-6918c399044f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.026976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93490-6170-4140-83ba-6918c399044f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.027246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c93490-6170-4140-83ba-6918c399044f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.027974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c93490-6170-4140-83ba-6918c399044f None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=42c1211f644d45bb8979455c4f28284a) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:21.028279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c93490-6170-4140-83ba-6918c399044f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.029735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c93490-6170-4140-83ba-6918c399044f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.035556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 284/572] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/role_assignments?user.id=42c1211f644d45bb8979455c4f28284a&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:21.036428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93490-6170-4140-83ba-6918c399044f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.040918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.041559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.041839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.042107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.054292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.055765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:21.072542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:21.073643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:21.074527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:21.074527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:21.074527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:21.074527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 289/573] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:21.074527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.097773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.146832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.157991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c93490-6170-4140-83ba-6918c399044f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.189235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced6a29-84ee-41ce-82d5-63e4dc3b1481 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.207983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.208645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.208922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.209191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.219935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.221468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.227243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.227846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.228123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.228787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.229214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:21.229511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.230957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:21.236607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:21.238148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:21.239034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:21.239034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:21.239034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:21.247109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 285/574] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:21.247109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.247321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 290/575] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:21.247321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.372234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.373984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728caf8f-acf5-467f-97b0-7b9dbc26a2c9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.375124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467f6b5-60dc-48b1-a7b1-f9e3f9e7159c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.402013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.411444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.414900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.415019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.415098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.415186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.415261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.415340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.415412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.415412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:21.415412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.415412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.423864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:21.428657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 291/576] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:21.429555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:21.430505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:21.431387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:21.432443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.433063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 286/577] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:21.433863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.542336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c701a50-8494-49b8-921e-4161a5e7c9be None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.548521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79478fe6-3f54-42f6-90aa-9c9bf852ccea None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.577906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.578513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.578799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.579067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.579709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:21.580009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.581455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.587658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.588290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.588595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.588869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.593093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 292/578] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:21.593699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.604639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.605964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:21.621116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:21.622099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:21.623008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:21.623008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:21.623008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:21.623008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 287/579] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:21.623008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.642019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.648567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.702806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb9140-41e9-4e47-8d23-b485ea40a36f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.735171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56d98c-ee85-458c-b922-384dd5802282 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.739462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.740088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.740394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.740669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.748194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.749653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:21.760899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:21.761841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:45:21.762794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:21.766951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 293/580] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:21.767537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.772777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.773388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.773667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.773939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.774608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:21.774911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.776527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.790210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 288/581] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:21.790210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.868304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.874821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc5b98-96cc-4158-9603-9155135ba385 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.882530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:21.896692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65045e2-b611-4c88-b3b0-e01f635b48c6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:21.910214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.910846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.911126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.911405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.918102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.920844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:21.931242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:21.932144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 294/582] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:21.933082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:21.937901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:21.938519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:21.938809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:21.939084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:21.939733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:21.940037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:21.941516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:21.958348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 289/583] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:21.959042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.039134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa209-4545-45f4-91b4-c4da3407af31 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.065207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808bacb-667b-4519-b596-a2ae4fcca8e9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.073741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.074602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.074877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.075148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.075883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:22.076188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.077644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.100365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 295/584] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:22.100365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.108812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.109454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.109731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.109995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.112095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.122013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.123447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.127943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:22.140674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:22.141910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:22.142817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:22.142817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:22.142817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:22.144663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 290/585] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:22.145671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.210212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934cae9-45b9-4c0d-a015-2fbe3a897727 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.249196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.249807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.250084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.250354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.251047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:22.251402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.252788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.254665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd0ac5-a414-49e2-86c3-a85eaa90f1cc None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.271296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 296/586] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:22.271296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.291317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.291938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.292222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.292526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.300561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.302007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:22.313430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:22.317384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 291/587] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:22.318343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.366127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.368455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.384000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e185079-85c2-46ac-bb3a-c8564f6c9b18 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.418797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.419467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.419725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.419997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.424077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9a9f-e490-49c0-a616-55e5452600d4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.431574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.433036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:22.449273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:22.450303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:22.451206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:22.451206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:22.451206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:22.451206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 297/588] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:22.451206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.465976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.466826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.467286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.467619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.468362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:22.468744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.470158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.486630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 292/589] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:22.486630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.560367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a500c63-06e3-412d-b178-adc137598ec1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.591440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e21e4-0fa3-41c1-b70c-4fb420129c19 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.595997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.596699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.596980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.597251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/99dec0395ef74fd5a6023ae06a1b9538/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.600528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.601842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.611994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=99dec0395ef74fd5a6023ae06a1b9538, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.613512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.629747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.630345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.630621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.630890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.631662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:22.631964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.633402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.651236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 293/590] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:22.651954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.681822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 298/591] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:22 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/99dec0395ef74fd5a6023ae06a1b9538/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:22.682841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.786588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777e77b-36eb-4cff-a7b7-fac6cf2a5e5d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.828902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.829381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.829611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f57bb-7d18-4f1e-82c6-2438572db01f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.829849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.830231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.830990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:22.831814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.833375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.847509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 294/592] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:22.848534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.879753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:22.883966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:22.884645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:22.889341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:22.889341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:22.894814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:22.896806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:22.912792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:22.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 299/593] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:22.915216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:22.967260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23914380-f38f-4572-8078-5f5c4962d7a8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:22.979128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.007007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.007648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.007931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.008202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.022441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.023904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.037575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d963a1-5a87-445f-8b62-78fefed4dca2 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:23.039975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:23.047399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:23.048614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:23.048614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:23.048614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:23.048614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 295/594] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:23.048614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.082218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.083004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.083377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.083743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.090628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.092108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f4a28cca-e50d-4db6-813f-b5c766a6c069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:23.103431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:23.104585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:23.105854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 300/595] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:23.106831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68e4b3-5230-4f7f-b640-5c336bd2ad46 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.107110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68e4b3-5230-4f7f-b640-5c336bd2ad46 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.107381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68e4b3-5230-4f7f-b640-5c336bd2ad46 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.113074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 301/596] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:23.113942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68e4b3-5230-4f7f-b640-5c336bd2ad46 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.133339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.156981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840576a-27cd-45fe-aba3-b69fdc4ad8dc None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.198665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.199363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.199646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.199919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.200696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:23.201017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.202414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.220991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 296/597] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:23.226315 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.230368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68e4b3-5230-4f7f-b640-5c336bd2ad46 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.238263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.270453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.271187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.271715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.272136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.273750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:23.274188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.275701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca940d4-a783-4bd6-8f7c-ccf71b48d186 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.298335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 302/598] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:23.298335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.298335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.298335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.317615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:23.318837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:23.356281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c8d46-054f-41c1-9601-ed2ead50d838 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.395207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.395819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.396074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 303/599] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:23.400814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.400814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.400995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.401081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:23.401081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.401081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.411372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.422747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 297/600] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:23.423658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.513846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.520680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652e380-1e35-47a2-b183-334202b1bd24 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.555134 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e6b7e-8b4a-4c9f-bcb6-41b5a02d1cd8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.558003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.558809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.559239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.559650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.570770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.572474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:23.588831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:23.590017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:23.591030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:23.591030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:23.591030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:23.592873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 304/601] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:23.593818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.600604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.601475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.601998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.602465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.610202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.611855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:23.624058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:23.625241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:23.626255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:23.628340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 298/602] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:23.629302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.690084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.729225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459984f9-82b3-4f81-9660-f3579fa85c55 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.747709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c160f-6b73-4353-a5ac-81706e2b680b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.765052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.765708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.765984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.766252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/9c32115f24264e738c432f700a2f7dcc/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.782488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=9c32115f24264e738c432f700a2f7dcc, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.783437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.783907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.784809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.785192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.785571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.786332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:23.786739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.788160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.800236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 299/603] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:23.800924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.853392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.868343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 305/604] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:23 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/9c32115f24264e738c432f700a2f7dcc/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:23.868343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:23.928130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788e6c-5694-4aec-a376-13ee5ea7f9c6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:23.945742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:23.969164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:23.969957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:23.970382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:23.970779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:23.971786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:23.972163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:23.974863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:23.996491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 300/605] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:23.998044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.026107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28b4e-67c1-4bec-a757-0cd7041462b3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.073554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.074201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.074483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.074760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.087969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:24.093682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:24.094597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:24.095438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:24.098412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 306/606] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:24.099464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.117550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1593a-db28-4223-adc9-1a9e25b603bd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.156193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.156860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.157140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.157410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.165519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.167100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.173253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:24.180180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:24.181140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 301/607] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:24.182436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.222643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99dc6-dba1-4339-b3ab-7a284298b7ed None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.229628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:24.261681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.262677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.263153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.263646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/dbb0ec082bdb45b0bb2c59cbad440d99/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.281378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=dbb0ec082bdb45b0bb2c59cbad440d99, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.283156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.314797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce3f9-e2bb-4641-a52a-b80ef9d260b3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.354411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.355021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.355299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.355572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.356246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:24.356580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.358026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.367415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 307/608] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:24 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/dbb0ec082bdb45b0bb2c59cbad440d99/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:24.368642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 302/609] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:24.369064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.381914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.527680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242942b-a87d-4933-9cbb-6de9b87892b3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.556425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:24.582968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.583589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.583870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.584210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.585072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:24.585385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.585851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:24.587743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.592528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d22435-8989-45ef-a8d4-4ad24b939950 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.603951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 308/610] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:24.610451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.638181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.639071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.639501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.639945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.659598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.662182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c8cca113-ec8f-4682-98ab-bb04dd816839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:24.690889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:24.692087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:24.693038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:24.693038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:24.693038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:24.693497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 303/611] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:24.694732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f173b3-f790-455f-8a98-f4a866663fa8 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.695079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f173b3-f790-455f-8a98-f4a866663fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.699779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f173b3-f790-455f-8a98-f4a866663fa8 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.704857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 304/612] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:24.704857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f173b3-f790-455f-8a98-f4a866663fa8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.749961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36091d90-5f9b-43f5-8ce7-862df6f0e3bc None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.786020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f50b8c-5140-43e2-ac1b-1b6451536436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.786756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f50b8c-5140-43e2-ac1b-1b6451536436 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.787042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f50b8c-5140-43e2-ac1b-1b6451536436 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.787457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f50b8c-5140-43e2-ac1b-1b6451536436 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/c16d573ee440426aaa09eddb9c2658dc/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.803378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f50b8c-5140-43e2-ac1b-1b6451536436 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=c16d573ee440426aaa09eddb9c2658dc, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.804920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f50b8c-5140-43e2-ac1b-1b6451536436 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.832139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f173b3-f790-455f-8a98-f4a866663fa8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:24.869225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:24.869812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.870097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.870366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.871014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:24.871461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:24.873000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:24.876118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 309/613] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:24 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/c16d573ee440426aaa09eddb9c2658dc/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:24.877791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d01041-da8e-423f-876e-153b380f400f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:24.882046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d01041-da8e-423f-876e-153b380f400f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:24.882046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d01041-da8e-423f-876e-153b380f400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:24.901938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 305/614] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:24.910983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:24.921487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-39d01041-da8e-423f-876e-153b380f400f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:24.925958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-39d01041-da8e-423f-876e-153b380f400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:24.937184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:24.958231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:25.060145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28f2f6-1c58-4c46-be91-46503c20e7e9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:25.091060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 310/615] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:25.101969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:25.102613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:25.102892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:25.103169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:25.111010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:25.112562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:25.124955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:25.125898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:25.126859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 306/616] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:25.132985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d01041-da8e-423f-876e-153b380f400f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:25.276183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d01041-da8e-423f-876e-153b380f400f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:25.311445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:25.312052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:25.312337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:25.312629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:25.313362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:25.313711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:25.315149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:25.333282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 311/617] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 1 17:45:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:25.338404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:25.442676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:25.488146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf524d-0857-4132-90c1-5e8e157bfb47 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:25.534161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:25.534758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:25.535038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:25.535509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:25.556537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:25.557999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:25.573289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:25.574314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:25.575380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:25.575380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:25.575380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:25.575380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 307/618] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:25.580803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:25.598965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:25.675169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:25.742952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182756b3-e459-4524-b8d4-d801a98e3bd1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:25.824454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:25.825547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:25.826040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:25.826504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:25.827379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:25.827886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:25.829464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:25.841713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 312/619] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:25.853273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:25.868693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab16f-daff-4f82-8fc7-2d147ea2b560 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:25.886954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:25.902841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:25.907102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:25.907102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:25.907102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:25.917120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:25.922446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:25.923380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:25.924259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:25.928051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 308/620] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:25.945746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:25.961489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:25.991926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aa4c2-93a3-4789-866e-44d2db58a5a5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.031703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.032349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.032638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.032911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.043613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.045084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:26.062594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:26.063609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:26.064924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:26.068345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 313/621] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:26.070963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.142495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453877b2-84b2-4b00-8171-6bc8da8de5eb None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.181957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.182560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.182836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.183117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.183863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bca2d-1648-44c1-9d7b-556e0b08d8fd None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.184005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:26.184162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.185563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50351582-38bf-419d-9689-cf2e70f90e65 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.194425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 309/622] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:26.200214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e660eec-cb69-4f33-8b4a-e97773e51432 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.200609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e660eec-cb69-4f33-8b4a-e97773e51432 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.200853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e660eec-cb69-4f33-8b4a-e97773e51432 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.206425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 310/623] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:26.206425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e660eec-cb69-4f33-8b4a-e97773e51432 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.227052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.227894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.228260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.228631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.229648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:26.230445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.232035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855e24e-578e-4dab-bd3e-d4ba9ea30779 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.242565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:26.251513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 314/624] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:26.251513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.251513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.251513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.272724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:26.274404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:26.275945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:26.356590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e660eec-cb69-4f33-8b4a-e97773e51432 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.374310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 315/625] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:26.374310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.441279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.442264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.442725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.443192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.462198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.464083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.483601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54de5e0-15df-4708-84f0-783ac5f5654d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:26.490911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:26.491882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:26.492815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:26.492815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:26.492815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:26.501988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 311/626] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:26.504777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:26.508368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.530001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9581a-b97f-42a7-948a-37bf9f238e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.530275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9581a-b97f-42a7-948a-37bf9f238e1e None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.530560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9581a-b97f-42a7-948a-37bf9f238e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.530959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9581a-b97f-42a7-948a-37bf9f238e1e None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/42c1211f644d45bb8979455c4f28284a/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.553720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9581a-b97f-42a7-948a-37bf9f238e1e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=42c1211f644d45bb8979455c4f28284a, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.555162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9581a-b97f-42a7-948a-37bf9f238e1e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.638348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 316/627] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:26 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/42c1211f644d45bb8979455c4f28284a/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:26.646112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:26.646505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070668e-2d85-4fea-8d0c-5a40753d153c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.647334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070668e-2d85-4fea-8d0c-5a40753d153c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.647807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070668e-2d85-4fea-8d0c-5a40753d153c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.649716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 317/628] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:26.654094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4473-d6a5-4660-aa3b-2d9bf8b9775c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.655797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.656255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.656723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.695904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.696458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.696789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.697065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.697466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:26.697771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:26.698137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.699293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:26.699656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.715176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 312/629] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:26.725343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.792047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 318/630] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:26.793460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.867859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee6cb5-cda1-43f5-aa4f-d69ff3554d26 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.912241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.912917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.913277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.913471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.921890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:26.923171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:26.932486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54782-cea5-4d65-baae-7fc87558a24b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:26.937422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:26.938371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:26.939418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 313/631] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:45:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:26.944675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:26.944836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:26.972500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:26.979716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:26.980676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:26.981158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:26.981653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:26.992896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:26.998348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:26.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:27.000170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:27.003420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 319/632] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:27.009607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.060140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e65f48-0dd7-45fc-9d10-6244fc08bc0c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.099297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.099954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.100239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.100543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.101341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:27.101647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.103084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.121662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 314/633] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:45:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:27.122815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcf2fa-98c2-4ab9-a608-2b370044ecb1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.130829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.174466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.175463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.176025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.176299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.177194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:27.177875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.179544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.191664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 320/634] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:27.192687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.224811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.230931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.270730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7a776-caaa-480c-bc56-56d0440253fe None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.308546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb261-0a07-4dff-abd3-17d34e1d57b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.309159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb261-0a07-4dff-abd3-17d34e1d57b7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.309441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb261-0a07-4dff-abd3-17d34e1d57b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.309612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcc286-219c-4b82-99e3-9ae3550bfd35 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.309714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb261-0a07-4dff-abd3-17d34e1d57b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.347975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.348638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.348952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.349185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.359855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:27.365270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:27.366214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:27.367067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 321/635] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:27.368629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.395578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffb261-0a07-4dff-abd3-17d34e1d57b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.397344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffb261-0a07-4dff-abd3-17d34e1d57b7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.451046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.471580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.473306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3b9d2-6d7f-40a7-bf31-f1551a3ee7bb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.502114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 315/636] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:27.504167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c475-5dfb-4961-8667-1ff510b47c3b None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.504454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c475-5dfb-4961-8667-1ff510b47c3b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.504730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c475-5dfb-4961-8667-1ff510b47c3b None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.506264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 316/637] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:27.506796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c475-5dfb-4961-8667-1ff510b47c3b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.517571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.518601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.518889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.519164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.531717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.533370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:27.552472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:27.553455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:27.554381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:27.554381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:27.554381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:27.554381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 322/638] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:27.555062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.617798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c475-5dfb-4961-8667-1ff510b47c3b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.652640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.653552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.653846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.654097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.654889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:27.655126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.656629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479627b7-73b3-4f23-81d8-2b0ee65ce7d9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.658721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb9715-4997-4fad-bc4e-329fa78a18cd None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.665712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 317/639] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:27.667120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.667532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.667944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.686987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:27.688168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:27.693832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.700065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.700720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.701038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.701312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.713965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.715919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.716621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:27.734771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:27.740863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:27.741791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:27.741791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:27.741791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:27.741791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 323/640] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:27.741791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.773171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 318/641] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:27.774039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.858559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f2cf1-41e7-48d2-920f-aa23669c30a7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.917664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.918277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.918555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.918827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.919477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:27.919778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.921174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:27.927114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d337b20-d968-4843-9f24-d626f515e983 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:27.934237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 324/642] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:27.935069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:27.965539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:27.966594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:27.967079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:27.967543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:27.980492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:27.981333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:27.983297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:28.010169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:28.011266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:28.012190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:28.012190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:28.012190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:28.014367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 319/643] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:28.015354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.036451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.091456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66455cd1-7a97-4e13-9c1b-b87dc1dc5a71 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.123099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e585f42-8ea9-46e3-88a9-b046ec7b966b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.131138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.132019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.132420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.132811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.133516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:28.133818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.135797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.152348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 325/644] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:28.153346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.162040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.162670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.162947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.163217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.171349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.172815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:28.184011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:28.185225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 320/645] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:28.186932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.279765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.290643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49917cb9-a35d-4a9d-b3d7-03ababbba015 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.292249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.304854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc487-451c-45c8-9b06-4b6bfe29c10a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.334198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.335153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.335614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.335891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.336613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:28.336924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.338335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.351575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 326/646] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:28.352218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.360540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.361207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.361487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.361758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.376843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:28.382554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:28.383621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 321/647] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:28.384506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.478204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530523dc-40a1-4fd0-8477-6f859d1d5f42 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.499142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b09a-9e10-4914-b02a-90c06147c697 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.518313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.519603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.519878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.520149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.528100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.531015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.537658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.538269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.538549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.538825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.539561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:28.539869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.541356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.543956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:28.544404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:28.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 327/648] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:28.546686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.566882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 322/649] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:28.567601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.578113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.665322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935ca48-3698-4e03-972e-912f0270b3bd None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.692766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207cf2d-c2a8-4e28-9956-4cd423501b92 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.711819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.712845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.713283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.713723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.724280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.725727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.733652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.734259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.734534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.734802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.735467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:28.735766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.737208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:28.738346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:28.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:28.740204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 328/650] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:28.740840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.748035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 323/651] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:28.748928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.826136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.835373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:28.854576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68cb7a-a5d1-489c-990f-ebd96f19da1f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.877380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7935c35-c7c3-4622-8a75-62b7a38dd06a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:28.887620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.888281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.888764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.889457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.890762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:28.891073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.892778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:28.904850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 329/652] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:28.905437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:28.919619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85f036-9609-4d89-bd08-d73d6acdcbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:28.920238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f036-9609-4d89-bd08-d73d6acdcbca None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:28.920547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f036-9609-4d89-bd08-d73d6acdcbca None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:28.920818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f036-9609-4d89-bd08-d73d6acdcbca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:28.991319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f036-9609-4d89-bd08-d73d6acdcbca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:28.992809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f036-9609-4d89-bd08-d73d6acdcbca None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.022942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef247a7-966b-43e4-99f9-7d844e60668c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.048744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.057457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.060794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.060794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.060794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.060794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:29.060794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.061202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.075542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.079028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 330/653] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:29.079837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.094476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 324/654] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:29.095753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fdc9b-2e0a-4d20-a2e3-f147156710fa None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.096042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fdc9b-2e0a-4d20-a2e3-f147156710fa None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.096344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fdc9b-2e0a-4d20-a2e3-f147156710fa None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.098736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 325/655] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:29.099517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fdc9b-2e0a-4d20-a2e3-f147156710fa None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.191220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e992a69-ff1a-42fc-a822-74727b1db9b4 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.202682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fdc9b-2e0a-4d20-a2e3-f147156710fa None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.233893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.234521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.234800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.235522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.248635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.250099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.255467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.256077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.256460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.256735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.257475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=99dec0395ef74fd5a6023ae06a1b9538) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:29.257784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.259211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.265421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 326/656] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/role_assignments?user.id=99dec0395ef74fd5a6023ae06a1b9538&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:29.266230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:29.267016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:29.267962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:29.269718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:29.269718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:29.269718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:29.272493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 331/657] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:29.273804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.293258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.345372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.395080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeefc88-7098-4440-a813-2659b1b3d9a3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.401910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b682c3a-9ac0-4259-9d6a-224c1a7c9121 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.443848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.444550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.444833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.445104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.446250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.446950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.447288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.447782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.466019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.467859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6cbeaa0b-3975-413b-8a2f-4f2c1bd3f701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:29.483935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:29.484990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:29.486115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:29.486115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:29.486115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:29.487209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 332/658] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:29.492530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.492909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.493252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.513921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:29.515148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:29.531449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.532941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.556243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.596909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 333/659] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:29.596909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.638138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.656294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 327/660] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:29.657714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.718351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefee7e7-a9a3-417b-95e0-a78a89a5a8a5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.756179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.756838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.757117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.757389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.758047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:29.758349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.759710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.775667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 334/661] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:29.776664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.792493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea021c-5581-4bab-80cf-b347a33d94ed None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.820746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.841587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.842439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.842929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.843341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.844595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:29.845233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.847398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.861071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 328/662] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:29.862053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.890191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:29.898027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111d4ea-515e-4b7d-b315-d5d8173c3132 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:29.935526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:29.936164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:29.936490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:29.936750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:29.947303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:29.948759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:29.965400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:29.966426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:29.967326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:29.967326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:29.967326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:29.971087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 335/663] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:29.972473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:29.975085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0280cdf-0bde-4814-84c0-19308a0fa8d4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.014458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.015190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.015372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.015610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.023735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.025545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:30.037987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:30.039029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 17:45:30.039983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:30.043736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 329/664] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:30.044666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.077346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f5f9-d7de-454b-a9e3-4051bf5be773 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.088769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.116102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.116679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.116956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.117221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.124830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.126247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:30.140690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:30.141570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:30.142725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:30.144510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 336/665] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:30.145428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.155310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8a9c-0df1-4e76-b6aa-1e5c61fd2259 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.156611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.193346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.193970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.194300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.194614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.205211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:30.210490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:30.211403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 330/666] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:30.217247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.261794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5a842-bfe8-4032-b1fa-7064a442ae81 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.298154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.298762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.299041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.299314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.300055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:30.300565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.301918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.318483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 337/667] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:30.318483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.331373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.335018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52707b-92b1-4baa-9f71-cdf700667691 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.370247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.370775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.371055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.371326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.372058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:30.372388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.373803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.386700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.395519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 331/668] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:30.400818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.427586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d55fa-34b2-4039-b0d7-764baa5d8a1a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.461991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.462602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.462877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.463151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.463843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:30.464149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.465589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.477783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 338/669] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:30.478613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.510288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b79b-dc90-4336-a6d7-3318ad93022b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.551921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.552571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.552850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.553120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.553855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:30.554160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.555521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.564269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 332/670] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:30.565107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.592555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.594332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781e67-e0cc-4c33-9a78-3ee549650da7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.623222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.632366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.633289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.641456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.641456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.644470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.646676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:30.664035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:30.665169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 339/671] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:30.666094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.684145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7077de-16b6-45de-ae88-7c3290a75631 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.724435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.725042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.725311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.725581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.777128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f741-25af-4d48-ad45-b6a2da268838 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:30.819158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.823571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.824239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:30.824878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:30.825238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:30.825508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:30.826324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=9c32115f24264e738c432f700a2f7dcc) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:30.826629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:30.828270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:30.840681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 340/672] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/role_assignments?user.id=9c32115f24264e738c432f700a2f7dcc&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:30.842632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:30.846369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.931133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:30.976546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 333/673] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:30.977806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.002109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628b5a-657b-40a4-8bbe-7c2efefeb561 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.046165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.046964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.047249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.047522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.061017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.062475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:31.083265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:31.084301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:31.085631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:31.085631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:31.085631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:31.085631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 341/674] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:31.085631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.136385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb455da-76f2-451d-a845-f4d76199a1a2 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.178476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:31.199345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.199882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.200161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.200480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.201759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:31.202149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.203593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.227631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 334/675] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:31.227631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.244448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f2bde-4f0c-4e5f-971b-a8b51a083ec0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.246464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:31.296774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.297652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.297934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.298207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.312785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.314517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:31.336826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:31.337847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:31.338739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:31.338739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:31.338739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:31.342493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 342/676] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:31.343335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.370997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787465fc-a696-4cca-a292-57c638bd3109 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.417879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.419739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.420224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.420720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.421575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:31.422073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.423670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.442849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 335/677] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:31.443987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.453239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5def6-e6b3-409a-9809-cf9c73ec18f8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.492867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.493550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.493834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.494110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.494902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=dbb0ec082bdb45b0bb2c59cbad440d99) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:31.495217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.497006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.499345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:31.504642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 343/678] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/role_assignments?user.id=dbb0ec082bdb45b0bb2c59cbad440d99&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:31.505872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.549421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:31.595149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a52eb3-d07b-4d28-a727-9c20913b5a3b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.640625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e837873-8815-475f-829e-116371ed969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.641570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837873-8815-475f-829e-116371ed969f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.642295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837873-8815-475f-829e-116371ed969f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.642794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837873-8815-475f-829e-116371ed969f None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.643908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e837873-8815-475f-829e-116371ed969f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:31.644467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e837873-8815-475f-829e-116371ed969f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.646195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e837873-8815-475f-829e-116371ed969f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.664661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 336/679] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:31.665292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aeead-4dd7-4d28-8821-5cce62ca1e10 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.666634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d1c19-ba9c-4494-863c-e85e823e346e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.667138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d1c19-ba9c-4494-863c-e85e823e346e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.667594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d1c19-ba9c-4494-863c-e85e823e346e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.669678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 337/680] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:31.670627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d1c19-ba9c-4494-863c-e85e823e346e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.723002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.723002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.723002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.723002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.732224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.733717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-a366a724-7082-443c-a90f-cfdbc64d27ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:31.746663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:31.747652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 344/681] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.749984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.750718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.769874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:31.771430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:31.823472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:31.829884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d1c19-ba9c-4494-863c-e85e823e346e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:31.869979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 345/682] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:31.871500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:31.888554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:31.889565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:31.890309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:31.890808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:31.894034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:31.906747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:31.908559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:31.932976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:31.933933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:31.934828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:31.937910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 338/683] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:31.940155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.113358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1958078-ab5f-494c-80e7-4a5630abe196 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.122415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31ffb7-7579-4b41-a442-da6f4654cad9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.160906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.161524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.161806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.162081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.163104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:32.163412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:32.164919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:32.184304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.185234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.185516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.185790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.186536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:32.186846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:32.186949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 339/684] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:32.187904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.188257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:32.211461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:32.212993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 346/685] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 1 17:45:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:32.213846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.297197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:32.310996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901262e-e53e-4d5c-9227-8805be789212 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.342552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ef9c6-862c-4c3e-9d86-22f6739abbde None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.347625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.348261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.348564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.348957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.361771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:32.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:32.369443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:32.370466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:32.374306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 340/686] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:32.375292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.387418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.388633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.389066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.389486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.390841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=c16d573ee440426aaa09eddb9c2658dc) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:32.391404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:32.393575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:32.401275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 347/687] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/role_assignments?user.id=c16d573ee440426aaa09eddb9c2658dc&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:32.408519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.507593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:32.530170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1b673-2e87-4825-8ba9-cd4d647237e8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.572888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.573378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.573654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.573920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.630752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ff26-777b-42c7-9207-35cfd76e4372 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.665255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:32.671255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:32.672449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.672743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:32.673523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.673972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.674420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.675797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:32.676288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:32.691592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:32.691592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 348/688] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:32.694775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.782922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:32.810602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 341/689] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:32.811406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c25a8d-9f6d-4049-8a05-c67b5745df22 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.818432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.857116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.857694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.857957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.858230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:32.870260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:32.871810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:32.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:32.890133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:32.891046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:32.891046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:32.891046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:32.891046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 349/690] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:32.899520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:32.946608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108b3b4-4eff-4bda-b6af-2c99b62fbc11 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:32.950757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:32.994995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:32.996027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:32.996624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:32.997154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.017860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a94e-b2c7-451e-9599-18aa319ffb20 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.020026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.022960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:33.051342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:33.052404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:33.053392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:33.053392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:33.053392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:33.062841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 342/691] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:33.069291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.072465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.073088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.073420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.073694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.074465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:33.074910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.076779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.095374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 350/692] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:33.103107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.108138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:33.203272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362be876-d035-404f-8e87-47b3476e80ba None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.213853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0ae2c-2fbc-4587-b011-8fceb1e6b5c2 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.216434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:33.245357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.245981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.246316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.246598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.247258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:33.247568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.249502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e3ac-9da5-4333-a92e-eecf685d7755 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.261250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.261878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.262163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.262438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.267712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 343/693] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:33.270265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664af8f-af93-4115-9314-7888082ee933 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.270757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664af8f-af93-4115-9314-7888082ee933 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.271182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664af8f-af93-4115-9314-7888082ee933 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.276633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.282796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 344/694] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:33.282926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.283021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664af8f-af93-4115-9314-7888082ee933 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-9802321f-0872-4710-b5d0-3b6f3a6b4edf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:33.291514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:33.292533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:33.293546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 351/695] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:33.294225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.298889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.300340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.318480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:33.319666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:33.384685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:33.395867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664af8f-af93-4115-9314-7888082ee933 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.403499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 352/696] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:33.404546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.487532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.488597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.489099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.489571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.509434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.511285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.528690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:33.531434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9f22b-95d2-4387-8d4a-4427af1342a2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:33.532536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:33.533452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:33.535787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:33.540427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 345/697] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:33.547482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.573777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.574784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.575073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.575355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.576114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:33.576456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.578257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.598551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 353/698] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:33.603986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.729296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2299-5b02-441b-b8be-665eb8b60da6 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.762008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f480bc-598c-4d69-a1d8-27bea5adbc7a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.769624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.770503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.771034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.771446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.799437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:33.804070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.804904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.805336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.805786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.806652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:33.806968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.808503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327212b-94ef-4dc6-b1c3-7a1f57b077b0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.826236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 346/699] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:33.826236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033f963-31a4-41a0-b40e-698341a4105a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.826236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033f963-31a4-41a0-b40e-698341a4105a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.826236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033f963-31a4-41a0-b40e-698341a4105a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.826236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 347/700] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:33.826236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033f963-31a4-41a0-b40e-698341a4105a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.847246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:33.862362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.863931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.941843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033f963-31a4-41a0-b40e-698341a4105a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:33.977407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 354/701] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:33.977407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:33.982458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:33.983148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:33.983433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:33.983706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:33.984300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:33.985685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-0dd0a8c6-0785-4f26-9c52-efa0f6a18014 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:33.997438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:33.998472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:33.999313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:34.004001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 348/702] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:34.005209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.005494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.005773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.025793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:34.027045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:34.034715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.086419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2f727-4fa1-46a1-b519-0996ba2ab5b9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.097849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 349/703] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:34.098879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.115632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.130156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.131070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.131739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.132461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.135233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=42c1211f644d45bb8979455c4f28284a) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:34.135233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.137198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.145635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 355/704] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:33 2022] GET /identity/v3/role_assignments?user.id=42c1211f644d45bb8979455c4f28284a&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:34.146535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.267894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa22f-c69d-4b2c-94fb-add4207cee34 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.286758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.306388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.307187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.307441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.307729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.308428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:34.308733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.310104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.316152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 356/705] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:34.322628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.358554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8872fa2-e08f-4e82-a7e0-14a36d12a0df None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.403269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.404995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.405786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.406265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.420647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.422128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:34.442869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:34.443904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:34.444910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:34.444910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:34.444910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:34.446025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 350/706] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:34.453246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.466341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033dd17-721f-4a5b-9450-f6c510aee19e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.518301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.519315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.520171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.520506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.521584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:34.521886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.523770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.532127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 357/707] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:34.533392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.539043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.584780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9bf43-33e1-4a3a-8825-901f4948ce05 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.601876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.649473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.650092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.650365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.650629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.651284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:34.651579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.652965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.691985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 351/708] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:34.693791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0de17a-ad21-4930-8527-8b6820c18f6a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.697488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.739622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.740903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.741653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.742536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.757685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.759722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:34.781189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:34.782208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:34.783127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:34.783127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:34.783127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:34.788536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 358/709] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:34.794228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.836260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d7689-3031-4075-8eff-ce7f73d0fe10 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.857212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.875904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.876965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.877451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.877989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.885941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:34.887879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:34.901542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:34.902511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:34.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:34.905942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 352/710] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:34.912246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:34.928551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:34.935734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f444cf0-8745-42f4-80a1-96f27d8336a1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:34.992173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:34.994757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:34.995602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:34.997893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:34.999889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:35.000217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:35.001636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb47904-1173-426a-8acb-911592ef112b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.016246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 359/711] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:35.018861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5666c3f-5bfa-4eec-89ea-7db42f4678c9 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.019360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5666c3f-5bfa-4eec-89ea-7db42f4678c9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.019854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5666c3f-5bfa-4eec-89ea-7db42f4678c9 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.025505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 360/712] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:35.027912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.028437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.028913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.047629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:35.049180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:35.070277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d715ca2-65b5-4e2b-946e-4cd0fbbe357f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:35.108807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:35.109423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.109707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.109982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.110681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:35.110989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:35.113702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.128597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 361/713] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:45:35.129905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:35.130716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 353/714] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:35.138750 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:35.200707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:35.298503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01341475-0a6f-40da-97bb-e031e304b250 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:35.312811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:35.341239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f8954-d337-44ab-b384-86d9657432b9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:35.368457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:35.369078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.369359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.369637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.370375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=99dec0395ef74fd5a6023ae06a1b9538) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:35.370684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:35.373033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.385527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 354/715] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:35 2022] GET /identity/v3/role_assignments?user.id=99dec0395ef74fd5a6023ae06a1b9538&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:35.400205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:35.400863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.401148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.401426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.402099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:35.403660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.459510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:35.493630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 362/716] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:45:35 2022] POST /identity/v3/users => generated 254 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:35.584698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8281-1c23-4b65-9bf1-5411d80fba5c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:35.590774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:35.631964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:35.633043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.633613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.634100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] PATH_INFO: `/v3/services/network` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.634901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:35.636571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:35.648402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:35.649468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:35.650342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:35.653519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 355/717] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:35 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:35.659820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:35.789246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0f96a-c8e4-484a-b801-78bc1a424f2f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:35.823826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:35.824467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.824746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.825020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.825650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] RBAC: Adding query filter params (name=network) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:35.825957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:35.827301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dee20-5f5a-47dd-a289-8f04e6773970 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:35.832570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 363/718] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:45:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:35.838563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:35.957970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e6cd4-bf43-47fc-99e5-82d85d38a8cf None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:35.997441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:35.998062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:35.998338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:35.998605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:35.999308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] RBAC: Adding query filter params (type=network) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:35.999608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:36.001013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3b790-fb92-4ca2-84e5-450b85656f5b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:36.008784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 356/719] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:45:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:36.635232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b905c0-e879-46aa-82cd-aea2cbbceb35 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:36.635757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b905c0-e879-46aa-82cd-aea2cbbceb35 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:36.635757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b905c0-e879-46aa-82cd-aea2cbbceb35 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:36.637445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 364/720] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:36.643728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e5e1d-80ce-4961-a0fc-9dc29ba6869a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:36.644177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e5e1d-80ce-4961-a0fc-9dc29ba6869a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:36.644571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e5e1d-80ce-4961-a0fc-9dc29ba6869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:36.668533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1a8e5e1d-80ce-4961-a0fc-9dc29ba6869a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:36.669741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1a8e5e1d-80ce-4961-a0fc-9dc29ba6869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:36.705933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:36.767206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 357/721] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:36.786522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e47856-3278-4784-9539-0fc860109ac1 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:36.786821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e47856-3278-4784-9539-0fc860109ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:36.787089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e47856-3278-4784-9539-0fc860109ac1 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:36.788456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 365/722] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:36.793539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:36.793834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:36.794114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:36.817708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:36.818916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:36.898492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 358/723] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:37.191308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e47856-3278-4784-9539-0fc860109ac1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:37.200970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:37.225738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:37.356302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e47856-3278-4784-9539-0fc860109ac1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:37.397012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:37.414049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0dc2-c53e-487a-a232-2839bfaa2c1c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:37.431310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:37.431927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:37.432208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:37.432548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:37.433211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:37.434639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:37.467459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:37.468011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:37.468293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:37.468602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:37.480118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:37.485257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:37.486959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:37.487905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 359/724] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:37.495806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:37.539612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 366/725] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:45:37 2022] POST /identity/v3/services => generated 251 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:37.624942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78906e83-6f6b-4648-b53d-689a606c7603 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:37.671335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:37.672001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:37.672218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:37.672526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:37.673203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:37.673508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:37.675123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd06778-9e22-49ed-aef2-30d999582cb0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:37.689848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 360/726] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:37.696689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:37.767288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ed823-9eaf-4142-8f0a-8e5e600dac6e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:37.767568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ed823-9eaf-4142-8f0a-8e5e600dac6e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:37.767784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ed823-9eaf-4142-8f0a-8e5e600dac6e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:37.771084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 361/727] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:37.775203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:37.775493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:37.775771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:37.803195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:37.804469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:37.806764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:37.810212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:37.849524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931f6d6-d162-4a95-b49e-cc36e8fae033 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:37.877975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 362/728] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 17:45:37.885884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:37.886523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:37.886802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:37.887075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:37.899043 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:37.900603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:37.916239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:37.917257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:37.918171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:37.918171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:37.918171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:37.918171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 367/729] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:37.923136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.046713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1b7e4-6153-40ac-bb76-51348bf80f84 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.083301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.083915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.084194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.084505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.085155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:38.085459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:38.086821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:38.094097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:38.099635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 363/730] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:38.105367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.191268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.220344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a04b0-a834-4c15-b7f0-ba07d3d51713 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.242303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:38.267423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.269062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.269527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.269982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.279381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:38.281211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:38.293896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:38.294920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:38.295920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:38.298562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 368/731] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:38.305354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.377322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02eb4-3b03-4a3c-98d1-5a5fe3edc776 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.416568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebd6f2-88fc-48b2-92a1-36143d7fc6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.417568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebd6f2-88fc-48b2-92a1-36143d7fc6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.418049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebd6f2-88fc-48b2-92a1-36143d7fc6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.418520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebd6f2-88fc-48b2-92a1-36143d7fc6a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.431188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebd6f2-88fc-48b2-92a1-36143d7fc6a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-2aebd6f2-88fc-48b2-92a1-36143d7fc6a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:38.438432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:38.439394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:38.440831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:38.443755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 364/732] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:38.445115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2a18e-5446-4fd2-8aca-a0da91291d68 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.445547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2a18e-5446-4fd2-8aca-a0da91291d68 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.445945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2a18e-5446-4fd2-8aca-a0da91291d68 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.447632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 365/733] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:38.453144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2a18e-5446-4fd2-8aca-a0da91291d68 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.465558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a83a5a-fb2f-4915-b53e-9a1c93069e2e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.511091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.511710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.512364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.512660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.513748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:38.514064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:38.515682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870854a8-be9d-48eb-9316-1df3fa896ecc None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:38.532911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:38.541613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 369/734] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:38.543124 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.543414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.548460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.567108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:38.568655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:38.587706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2a18e-5446-4fd2-8aca-a0da91291d68 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.609870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:38.625701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.626591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.627042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.627500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.628635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:38.629342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:38.631900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:38.644380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 366/735] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:38.645707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.663324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 370/736] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:38.664476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.770828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999075-531b-481e-a76f-b654ec50eb8c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.781869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c54317-5714-4ce3-99a9-21d8535955f9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:38.794245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:38.810143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a24b0-c49a-4eff-9cef-db3e7fc3def6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.811095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a24b0-c49a-4eff-9cef-db3e7fc3def6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.811580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a24b0-c49a-4eff-9cef-db3e7fc3def6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.812053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a24b0-c49a-4eff-9cef-db3e7fc3def6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.826002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:38.828554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.828862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.829243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.843766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:38.845473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-da9cb144-83c9-4803-b89a-2dcec2cbd4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:38.866439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:38.867530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:38.868482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:38.868482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:38.868482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:38.872420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 371/737] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:38.873929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65d0e1-dedf-4295-9a28-2cfab4b19f63 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:38.874235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65d0e1-dedf-4295-9a28-2cfab4b19f63 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:38.874511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65d0e1-dedf-4295-9a28-2cfab4b19f63 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:38.876447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 372/738] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:38.877160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65d0e1-dedf-4295-9a28-2cfab4b19f63 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:38.897754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a24b0-c49a-4eff-9cef-db3e7fc3def6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:38.899640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a24b0-c49a-4eff-9cef-db3e7fc3def6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:38.904169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.021784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 367/739] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 17:45:39.023388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.023875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.024400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.027438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65d0e1-dedf-4295-9a28-2cfab4b19f63 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.044524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:39.046735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:39.065467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.068269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.069075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.069503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.069949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.070744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:39.071192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.072718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.085166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 373/740] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:39.085986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.125180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 368/741] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 17:45:39.126020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.181924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.210614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3aa43-4811-4493-94f2-b505965d0582 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.247758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.248266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.248589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.248859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.264085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:39.269439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfb11-03b9-4253-97db-90f4961bccb1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:39.273462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:39.278301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 374/742] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:39.279160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.319862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.320947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.321377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.322133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.339739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.339739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:39.357753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:39.358922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:39.359825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:39.359825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:39.359825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:39.362354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 369/743] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:39.363736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.367330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.417583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9d7ca-4c77-4116-a18a-6d6ca9e11f96 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.455714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.456365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.456649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.456927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.464760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.466195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.468525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.476604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171423f-c845-449c-9391-4f81cfb6a141 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:39.479343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:39.484618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 375/744] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:39.485586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.516751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.517618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.517893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.518197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.518948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:39.519167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.520596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.529080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 370/745] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:39.534286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.602567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7fefd-d364-424d-8200-42906be7b82a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.608925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.638917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.639525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.639799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.640070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.640748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:39.641053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.642690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.653326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692f9c6-4b14-433d-8241-eb42ae07d514 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.654626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 376/746] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:39.655415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.697441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.698321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.698321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.698607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.699251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.700726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.717374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.757549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9cb7c-87f6-4cf7-8b09-8e2c01d1c298 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.780542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 371/747] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:45:39 2022] POST /identity/v3/services => generated 240 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:39.780542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.835955 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.837299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.837893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.838344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.839281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:39.839831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.841521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.862920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 377/748] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:39.864339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:39.914228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35f229-48b2-4d26-8605-b55c7c61074a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.946140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:39.959621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:39.960272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:39.960614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:39.961227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:39.973919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:39.975476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:39.991860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b883-096f-40d0-8e52-2da13ab080b0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:39.992945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:39.993876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:39.994763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:39.994763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:39.994763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:39.994763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 372/749] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:39.998715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.010652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.033241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.033961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.034199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.034470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.042425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.043902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-212b67c3-077a-4644-ad77-6c657eae86bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:40.055792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:40.056773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:40.057695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 378/750] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:40.058654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d24115-6713-47f6-b9b4-723aa79fb82f None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.058751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d24115-6713-47f6-b9b4-723aa79fb82f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.058952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d24115-6713-47f6-b9b4-723aa79fb82f None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.060337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 379/751] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:40.060895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d24115-6713-47f6-b9b4-723aa79fb82f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.115675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3fcf4-f5c8-4f5d-9aba-40f4ae8ddb54 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.152277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.152934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.153215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.153487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.164378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.165825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.172808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d24115-6713-47f6-b9b4-723aa79fb82f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:40.180735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:40.181703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:40.182644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:40.182644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:40.182644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:40.182644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 373/752] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:40.182644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.214298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.218283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.218848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.219128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.219398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.220052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:40.220377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.221795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1be53-38d1-44cf-a269-1610b834c3d7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.233994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 380/753] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:40.239503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.239796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.240076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.259794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:40.261036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:40.290644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.306432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4eb5e-08a0-4adb-8aea-32b29c781cc4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.336699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 381/754] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:40.337552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.352064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.353256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.353740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.354304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.355228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:40.355733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.357513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.371699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 374/755] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:40.372896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.469372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccba4c-a051-4f3a-9726-caee2d54dd81 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.472090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.484163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da5604-ce10-4463-a4ba-b845c4bd4ec6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.520038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.521001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.521483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.521953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.522220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.522581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.522858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.523124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.523716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:40.524215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.526670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.530402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.531834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:40.545457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:40.546424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:40.547864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:40.549223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 382/757] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:40.550277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 375/757] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:40.550987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.551108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.585927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.670505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac377a6-e730-40a7-a55c-aedf2617203b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.674830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863184-282d-400a-a6f8-cb27bb8cba1a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.708971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.709574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.709858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.710131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.710862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=42c1211f644d45bb8979455c4f28284a) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:40.711168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.712611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.713893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.714702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.715150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.715614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.718546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 383/758] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/role_assignments?user.id=42c1211f644d45bb8979455c4f28284a&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:40.719346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.725392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.727360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:40.743059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:40.744052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:40.745004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:40.753611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 376/759] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:40.753611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.770352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.840437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:40.856525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d2ee1-b792-4ee3-8aaf-12d419c76942 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.893068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc0979-a7f9-4fbd-88a2-561b69bc3d64 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:40.904643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.905673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.906129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.906577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.907494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:40.908045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.909916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.929837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:40.930395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:40.930671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:40.930941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:40.931628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:40.932080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 384/760] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:40.932997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:40.933400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:40.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:40.944158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:40.945671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:40.949310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 377/761] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:40.950177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.023685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.048895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217fe68-4f0c-4197-aa4f-c7244fc2613e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.054522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca4526-ce94-4d31-9a44-b9918e121fdc None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.088221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.089130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.089407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.089683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.090512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:41.090819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.091167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.091943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.092222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.092535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.092944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.105670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 385/762] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:41.106680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.110745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.164440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.166160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.220726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5a6c-deab-45b5-866f-33b4ef243e30 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.244268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.264069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.264906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.265336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.265836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.266688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:41.267516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.269294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.275557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 386/763] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:41.276592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.287113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 378/764] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:45:41.288040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.361512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.392768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787abc81-8212-413a-ad44-58f81a90ca2f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.400628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50589c7d-95a1-4db3-910b-6153ce10c904 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.427949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.428608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.428890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.429218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.429782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:41.430081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.431446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.440503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 379/765] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:41.441598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.443877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.444754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.445387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.445900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/99dec0395ef74fd5a6023ae06a1b9538/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.448172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=99dec0395ef74fd5a6023ae06a1b9538, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.449826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.480042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.526839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 387/766] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:41 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/99dec0395ef74fd5a6023ae06a1b9538/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:41.528346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.549146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c9199-5e12-438a-873d-36b615948be5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.590129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.590791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.591096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.591368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.609016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.610585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:41.628554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:41.629966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:41.631255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:41.631255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:41.631255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:41.631255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 380/767] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:41.636921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.674875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fabe12-767e-4b12-98ff-9c4cfc7a8912 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.704301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.713441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.721921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.721921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.721921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.721921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] RBAC: Adding query filter params (interface=public, service_id=68c018b66c0349c995a19418f79e879b, region_id=RegionOne) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:41.721921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.721921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.725716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 388/768] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/endpoints?service_id=68c018b66c0349c995a19418f79e879b&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:41.731413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.762659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:41.780597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aaae8-8ee8-4697-8b9e-16f6a7e7516e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.825570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.826411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.826689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.827791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.828480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:41.828786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.830155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1743147-debe-46f6-adb2-90ff5b413b50 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.849345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 381/769] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:41.851158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bee9d-9838-4081-a301-a66a130651d0 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.851730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bee9d-9838-4081-a301-a66a130651d0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.852000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bee9d-9838-4081-a301-a66a130651d0 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.857435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 382/770] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:41.858214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bee9d-9838-4081-a301-a66a130651d0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:41.868082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6f7e-0694-4cc0-9117-b21f9f81b76a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:41.904103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d29dc1-24ba-4163-ab80-c0003a296933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:41.904744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d29dc1-24ba-4163-ab80-c0003a296933 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.905029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d29dc1-24ba-4163-ab80-c0003a296933 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.905285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d29dc1-24ba-4163-ab80-c0003a296933 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.905954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d29dc1-24ba-4163-ab80-c0003a296933 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:41.907614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d29dc1-24ba-4163-ab80-c0003a296933 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:41.915614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 389/771] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:41.926857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:41.927162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:41.927386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:41.954058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:41.955980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:41.996124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bee9d-9838-4081-a301-a66a130651d0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.038766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.039424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.040284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.040591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.059547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.061100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.084704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:42.090394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:42.094242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:42.095157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 383/772] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:42.098302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:42.121078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:42.123061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 390/773] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:42.216296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eec1c-2d97-4571-80d6-03d7b0cafc21 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.255137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.255672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.255947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.256215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.256948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:42.257249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.258781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.272763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 384/774] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:42.278447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:42.475073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:42.511421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b3ffb-de60-44cc-a98a-ef554452d806 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.565651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.566634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.567116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.567593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.569085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=16dc787e53484e0ea5ab6757d99c8a83) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:42.569611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.571222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.577967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 391/775] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:42 2022] GET /identity/v3/role_assignments?user.id=16dc787e53484e0ea5ab6757d99c8a83&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:42.622695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:42.676507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86e8b-2370-4470-b6fd-eca5da1acb55 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.727516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.728175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.728505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.728781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] PATH_INFO: `/v3/services/network` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.729378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.730780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f074df0b-68a2-4092-98a3-5b3572102048 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:42.746337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:42.747461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:42.748340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 385/776] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:42 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:42.756133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:42.819216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:42.883647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3652-a5f0-4310-a801-996a9b22031a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:42.911878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e243c6-8d39-4695-8d73-6bda16027756 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.912221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e243c6-8d39-4695-8d73-6bda16027756 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.912472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e243c6-8d39-4695-8d73-6bda16027756 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.914038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 386/777] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:42.919451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.919731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.920010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.942010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:42.942634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:42.942912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:42.943182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:42.943826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] RBAC: Adding query filter params (name=network) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:42.944126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:42.945589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:42.954139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:42.955340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:42.959704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 392/778] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:45:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:42.966791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.034461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 387/779] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:45:43.149198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5a109-4a25-4954-9662-260c9843f40c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.176755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:43.201019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.201633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.201912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.202188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.202824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] RBAC: Adding query filter params (type=network) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:43.203122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.204548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.209549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 393/780] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:45:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:43.215531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.280653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:43.321890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247dedd-6a6d-479b-9779-f529b2888b89 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.357481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.358087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.358361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.358630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.359327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] RBAC: Adding query filter params (interface=public, service_id=8b2a0e142f5e4e6186012c6e6899917b, region_id=RegionOne) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:43.359630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.360996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.369620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 388/781] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/endpoints?service_id=8b2a0e142f5e4e6186012c6e6899917b&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:43.375164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.424829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.500074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace83e96-87f6-4f6e-a995-7a99e10e5045 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.539429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.540041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.540338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.540615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.541222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.542560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.555167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 394/782] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:43.620634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abd38a-1a31-4ee0-afc4-7758f512be94 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.671210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:43.691149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.691149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.691149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.691149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.702325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:43.708088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:43.709162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:43.711073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 389/783] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:43.716424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.797939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:43.857546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1bdb6-1c2b-44c2-a444-1948d4e60c42 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:43.898026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:43.898646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:43.898930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:43.902676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:43.902676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:43.902676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:43.902676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723834b2-4533-418a-bf9f-e194de052e54 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:43.912185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 395/784] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:43.917569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:43.999903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e235838-6037-4c67-a606-b184050d5ada None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.000745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e235838-6037-4c67-a606-b184050d5ada None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.001407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e235838-6037-4c67-a606-b184050d5ada None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.007364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 396/785] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:44.009449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.010161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.010822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.042242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:44.043517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:44.145678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df707e-1f51-4b49-b0c4-4f329f70c247 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:44.175760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 397/786] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:44.202396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:44.203348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.203826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.204328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.215942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:44.217814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-b5ad1b03-b8ba-4db9-9de6-7023a03c99f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:44.238998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:44.240015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:44.245363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:44.245363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:44.245363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:44.247372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 390/787] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:44.253381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:44.304511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:44.415190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d631f9-aeec-4bc2-85fe-efe2ab0f6590 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:44.451906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffbe1d-6367-44a3-9a27-fe87435de8ca None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.452601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffbe1d-6367-44a3-9a27-fe87435de8ca None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.452880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffbe1d-6367-44a3-9a27-fe87435de8ca None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.454440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 391/788] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:44.460303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.460582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.460846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.466388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:44.467327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.467800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.468378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.469366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:44.469671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:44.471142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:44.481514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:44.483024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:44.489476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 398/789] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:44.496270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:44.540637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:44.600348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 392/790] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:45:44.600348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:44.655836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:44.675059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf1e2d-2cbe-455e-b431-ef26130e2a41 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:44.714390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac29e1-2288-467b-a5e4-abf9675992d9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:44.756268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:44.757245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.758036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:44.758525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.759322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:44.759766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.760702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:44.761167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:44.774925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:44.780629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:44.781988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:44.787336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:44.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:44.789139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:44.797338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 393/791] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:44.803787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:44.805112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:44.806241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:44.807387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:44.818332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 399/792] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:44.823942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:44.866900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:44.970887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81e42e-bdef-4a99-b4e2-d8cc46f2ed1f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:44.977222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d089-3eb1-49b4-91d8-49e4b58a11f4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.011149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.011816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.012039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.012324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.013095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:45.013385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.014861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590a1cc-974d-4661-bbec-0046dc158c7b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.016356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.017377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.017797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.018815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.019912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:45.020211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.021590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.030384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 394/793] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:45.031257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.036437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 400/794] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 1 17:45:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:45.037933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5583-56c2-470e-a239-a61da93e0a2f None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.038396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5583-56c2-470e-a239-a61da93e0a2f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.038853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5583-56c2-470e-a239-a61da93e0a2f None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.040591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 401/795] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:45.041466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5583-56c2-470e-a239-a61da93e0a2f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.057123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.125236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.136305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6facf8-d14b-44ff-8ea1-fc02efa77a3b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.156148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5583-56c2-470e-a239-a61da93e0a2f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.172244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.173018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.173296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.173564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.174343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.175788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:45.185453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:45.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:45.187389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 395/796] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:45.188819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.196027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.196705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.196981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.197253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.208214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.209952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-57ade297-b927-4ca5-ab67-bdb23a90b5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:45.224687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:45.225826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:45.226715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:45.226715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:45.226715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:45.226715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 402/797] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:45.227200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.227478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.227745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.247389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:45.248590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:45.314243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.318282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c428-8de9-4752-b74e-4598caa34aca None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.324159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 403/798] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:45.326062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.392544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.393368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.393732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.394002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.407507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.409553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:45.427672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:45.428694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:45.429594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:45.429594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:45.429594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:45.429594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 396/799] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:45.429594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.440208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.472556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c57b0c-dd2f-44d9-8706-1c65c98349f3 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.509872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.510809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.511289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.511730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.512557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:45.513128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.514669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.524855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 404/800] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:45.526008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.540826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aae26e-873c-4ee7-b8d6-81e453c5b4eb None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.576425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.576959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.577233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.577501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.578238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:45.578532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.579892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.592079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.594032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 397/801] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:45.595010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.639471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d659-7df9-498f-b018-cac2a63372e7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.668402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.676587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.677198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.677470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.677738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.678379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:45.678676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.680011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.692127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 405/802] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:45.692809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.711791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbae86-57fe-44ea-b893-9906d6551c3d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.749532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.750144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.750428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.750703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.751340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:45.751645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.753422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.762973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 398/803] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:45.762973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.801874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3deaf84-b58c-4faf-aedd-256919bdaad1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.819193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.846653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.846653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.846653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.846653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.847062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:45.847062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:45.857974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:45.858944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 406/804] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:45.859932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.867183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4403f-f6d6-4a1e-8d7f-badbbf75f09a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.897470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:45.903260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.908759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.908759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.908759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:45.914865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:45.919933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:45.921087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:45.922131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:45.929454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 399/805] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:45.930289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:45.961395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85b39-7c49-4af5-a4d7-8af274ac44c6 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:45.994479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:45.995106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:45.995387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:45.995667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.002732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.004733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-fcad59e7-764f-4754-b215-921df3beb8c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:46.015856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:46.016884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:46.018605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 407/806] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:46.019265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3d02-5382-4347-9f48-7464c5352b16 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.019622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3d02-5382-4347-9f48-7464c5352b16 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.019902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3d02-5382-4347-9f48-7464c5352b16 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.021482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 408/807] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:46.022212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3d02-5382-4347-9f48-7464c5352b16 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.034074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ce42f-1a8a-424e-9e0d-7cc2fb3c7662 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.062810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.069653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.070257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.070536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.070806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.071466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.072873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.122628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3d02-5382-4347-9f48-7464c5352b16 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.156064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.156682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.156961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.157230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.157948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:46.158248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.159677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.176568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 409/808] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:46.177339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.224419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 400/809] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:45:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:46.225297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.284763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.289832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.298301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94834db1-e7c4-4b89-8355-28256b202cc2 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.342103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.342729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.343007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.343279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.343958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:46.344260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.345967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcd60b-bfcf-420c-a2d4-fe4ad7e132ef None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.346232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d48ba-99f8-4f26-95d2-2394354b623e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.355801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 410/810] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:46.357306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.357592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.357869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.380640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:46.383017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:46.393576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.394500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.394922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.395195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.395904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:46.396236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.398561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.416581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 401/811] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:46.417311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.464471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 411/812] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:46.466267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.530308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425461d4-5930-46c4-a55f-8245ac411842 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.576197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.594775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a735ea-38f9-4f1b-938a-d3b554b09530 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.634266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.635253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.635714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.636167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.648431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.650214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.657749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:46.670924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:46.675485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:46.676447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:46.676447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:46.676447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:46.676892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 412/813] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:46.677703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.693008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.693953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.694592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.695014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.771742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.773239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.797571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd48f3-486f-4104-8fe4-bc91bc9b4bee None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:46.832897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:46.833554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:46.833834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:46.834106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/42c1211f644d45bb8979455c4f28284a/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:46.835992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=42c1211f644d45bb8979455c4f28284a, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:46.837430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:46.840783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.876424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 402/814] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:45:46.878030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.886040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:46.932385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 413/815] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:46 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/42c1211f644d45bb8979455c4f28284a/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:46.933181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:46.996183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0871a7-fd51-4543-a775-1311d24ce8ec None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.051874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.052522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.052804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.053076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.053734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:47.054045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.055477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.058546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8b857-d702-46b3-b9f1-1e08c00a7b76 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.073909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 403/816] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:47.074894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.100295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.101074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.101262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.101482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] PATH_INFO: `/v3/services/network` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.102489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.103398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:47.118173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:47.119379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:47.120689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 414/817] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:46 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:47.121552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.155343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.193802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc9e96-8d16-4781-af2a-73fc61af5705 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.208716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.231844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.232887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.233169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.233721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.257161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.258944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.263199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790f8ee-2b7d-43fc-ab7d-e91482a6c929 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:47.279394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:47.285865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:47.286777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:47.286777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:47.286777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:47.286777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 404/818] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:47.286777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.306342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.307374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.307954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.308866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.317527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.319437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:47.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:47.334752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:47.335794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:47.342945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 415/819] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:47.344076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.400806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832ad0-fb0a-421b-90b6-27342b9cf833 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.419037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.436885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.439816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.439816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.439816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.439816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] RBAC: Adding query filter params (name=network) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:47.439816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.440566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.445617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 405/820] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:47.446532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.458765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4116a8-1bb4-4f7d-93a1-3f2bb9e5cb5e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.482473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.497726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.498565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.498995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.499405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.500187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:47.500663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.502155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.514824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 416/821] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:47.515967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.575073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8206220-c910-42da-a603-8cea9f553077 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.619714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.620603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.620891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.621164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.622391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:47.622702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.624438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.645253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3260c-5254-4a43-be17-f5695b63efb1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.649706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 406/822] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:47.650664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.689266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.689947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.690228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.690690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.691339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] RBAC: Adding query filter params (type=network) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:47.691644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.693019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.696705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.706378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 417/823] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:47.707300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.758468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.759005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46d0c-d6dd-44fe-af96-b649093695fa None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.795640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.796652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.797106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.797572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.804893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.806560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:47.818219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:47.819132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:47.820037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:47.820563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8c4dd-21ed-4dfe-956c-f690751d4033 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.822353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 407/824] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:47.823326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.860685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.861507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.861736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.862014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.873205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.874770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:47.891162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:47.892669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:47.894523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:47.894523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:47.894523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:47.894523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 418/825] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:47.894523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:47.936388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:47.939472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e87a46-3320-4771-b90e-5dadac1e08d5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:47.975482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48171502-061d-4d21-9e78-788047fec738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:47.976473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48171502-061d-4d21-9e78-788047fec738 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:47.976924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48171502-061d-4d21-9e78-788047fec738 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:47.977409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48171502-061d-4d21-9e78-788047fec738 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:47.978226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48171502-061d-4d21-9e78-788047fec738 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:47.979755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48171502-061d-4d21-9e78-788047fec738 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:48.003510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:48.009587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a17fe-464c-482a-9180-3c6851a48453 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:48.057093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 408/826] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:45:47 2022] POST /identity/v3/endpoints => generated 337 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:48.058583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48171502-061d-4d21-9e78-788047fec738 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:48.073321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:48.079709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.080154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.081352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.082290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:48.082731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:48.084602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:48.115593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 419/827] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:48.125849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:48.294551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48171502-061d-4d21-9e78-788047fec738 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:48.340618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:48.340835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d7f22-25e5-439f-8cd0-7b071510a45f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:48.341245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.341522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.341797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.342446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:48.342944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:48.345248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb68b89-a5a1-422a-8525-19fa002c6299 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:48.362214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 409/828] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:48.363794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec18c-cf06-4fe5-8659-d03959ac4f7d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.364096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec18c-cf06-4fe5-8659-d03959ac4f7d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.365007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec18c-cf06-4fe5-8659-d03959ac4f7d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.366545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 410/829] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:48.368524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ec18c-cf06-4fe5-8659-d03959ac4f7d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:48.392712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:48.396841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.396841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.396841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.396841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=99dec0395ef74fd5a6023ae06a1b9538) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:48.396841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:48.397544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b2528-6122-4740-8034-945cb46e02c1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:48.400821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:48.404820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 420/830] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:48 2022] GET /identity/v3/role_assignments?user.id=99dec0395ef74fd5a6023ae06a1b9538&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:48.406414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.406608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.406864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.438603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:48.441033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:48.468725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:48.559569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 421/831] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:48.603968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ec18c-cf06-4fe5-8659-d03959ac4f7d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:48.654454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:48.654454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.654808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.654808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.665286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:48.666767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:48.679543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:48.680509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:48.681964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 411/832] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:48.686305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:48.798133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c77205-c00d-40d2-9975-4d4cc64de02e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:48.834912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:48.835444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:48.835725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:48.835995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:48.836722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:48.837024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:48.838433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:48.851416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 422/833] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:48.853301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:48.858036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:48.868722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:48.958341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:48.966687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ae49d-4111-4f1b-9a5c-6ef8534835e9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:48.990798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17b502-afa5-4670-9fc4-f8e19f10d62b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:49.006358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d1575-5a07-473e-8480-6ef5c0e58c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:49.007350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d1575-5a07-473e-8480-6ef5c0e58c66 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.007802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d1575-5a07-473e-8480-6ef5c0e58c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.008247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d1575-5a07-473e-8480-6ef5c0e58c66 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/16dc787e53484e0ea5ab6757d99c8a83/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.026013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d1575-5a07-473e-8480-6ef5c0e58c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=16dc787e53484e0ea5ab6757d99c8a83, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:49.027423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d1575-5a07-473e-8480-6ef5c0e58c66 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:49.034606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:49.035234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.035544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.040940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.040940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:49.040940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f6fd0142-4541-4f43-ad0e-72cd89f014bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:49.046614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:49.047824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:49.049568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 412/834] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:49.050600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b79f79-007a-4bf5-9a73-733911aefe77 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.050600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b79f79-007a-4bf5-9a73-733911aefe77 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.050600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b79f79-007a-4bf5-9a73-733911aefe77 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.052346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 413/835] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:49.053722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b79f79-007a-4bf5-9a73-733911aefe77 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:49.130005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 423/836] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:45:48 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/16dc787e53484e0ea5ab6757d99c8a83/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:45:49.131344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.131700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.132053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.151824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:49.153176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:49.202420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b79f79-007a-4bf5-9a73-733911aefe77 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:49.218410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:49.239428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 424/837] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:49.255010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:49.259177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.259819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.260386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.261302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:49.261771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:49.263456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:49.270070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 414/838] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:49.276025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:49.392422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000006-8940-4205-8d17-d52ddc5dc7b5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:49.433039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:49.434014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.434471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.435114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.435973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:49.436499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:49.438391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:49.444649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 425/839] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:49.534063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:49.539247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:49.645770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe88429-7bf6-45d6-a136-6b185a652e37 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:49.681840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:49.682848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.683269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.683710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.696893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:49.703125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:49.704257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:45:49.706022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:49.712431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 415/840] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:49.714618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:49.773697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:49.829174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a331e-e0dc-4858-ba80-6e79457a5a6c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:49.865571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:49.866350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:49.866721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:49.867086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:49.867835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:49.868233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:49.869677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:49.878836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 426/841] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:49.884345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:49.952465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:49.993708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f00e6-0446-4f8f-9615-7d5e70d4ce07 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:50.029829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:50.030615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.030891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.031160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.043247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:50.044705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:50.059647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:50.060756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:50.061670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:50.061670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:50.061670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:50.061670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 416/842] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:50.069888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:50.191238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af07915-f19c-4312-a3f0-48acaa977f8f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:50.204810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:50.226887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:50.227486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.227759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.228082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.228771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:50.229069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:50.230405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:50.248387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 427/843] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:50.250192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:50.364592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f9eec-2583-4783-90f9-6bf6fbbfab8f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:50.381636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:50.401595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:50.402154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.402561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.402830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.410784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:50.412214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9a361541-0c44-4e1f-958b-a047082663d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:50.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:50.425146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 17:45:50.426092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:50.430021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 417/844] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:45:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:50.431433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:50.543712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ab7be-19df-4126-856b-7a671e2350f4 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:50.553974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c3c8-845d-4954-83e3-28a9b8aa4178 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.554472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c3c8-845d-4954-83e3-28a9b8aa4178 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.554595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c3c8-845d-4954-83e3-28a9b8aa4178 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.556091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 418/845] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:50.560995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.561275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.561550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.580842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:50.582152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:50.583965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:50.584824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.586012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.586297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.587035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:50.587334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:50.588770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:50.607957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 428/846] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 1 17:45:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:50.615332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:50.666283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:50.669702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 419/847] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:45:50.729610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194c34f-d9fe-49d3-9d44-863ec3421680 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:50.767853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:50.769035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:50.769508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:50.769954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:50.839309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:50.841047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:50.870721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:50.944764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 429/848] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:45:50.953476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:50.955641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.067014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0178a-8b80-42d5-8010-bda4b8674a1a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.074828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f366ad-d296-44b9-a868-732baaa13e51 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.075587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.105159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.106135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.106311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.107184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.107775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.114463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.115334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.115756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.116171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.117233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.118944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.121204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.122821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:51.129237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:51.130197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:45:51.131108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:51.133801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 430/849] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 1 17:45:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:51.139195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:51.140172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:51.141151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:51.141151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:51.141151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:51.145321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 420/850] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:51.145777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.146791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.259915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa320285-ce52-4357-b786-1406990af6e0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.264211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3a448-4caf-43e4-ae0b-b0f47c667d8f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.299263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.299863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.300138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.300439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.301081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:51.301379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.301935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.302570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.302753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cca1df-8f73-4af7-80b2-963c77229b17 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.302846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.303139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.303768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:51.304077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.305476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefb688-4d30-4a7a-a141-9ec7e993f004 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.309983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 431/851] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:51.311142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a339118-0f55-46b1-89f4-77046f7e7fa2 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.311423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a339118-0f55-46b1-89f4-77046f7e7fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.311697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a339118-0f55-46b1-89f4-77046f7e7fa2 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.313155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 432/852] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:51.315660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a339118-0f55-46b1-89f4-77046f7e7fa2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.324105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 421/853] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:51.325745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.326187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.326655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.349911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:51.351384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:51.368667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.381521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.425863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a339118-0f55-46b1-89f4-77046f7e7fa2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.433738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 422/854] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:51.434818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.461464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.462125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.462402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.462672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.463355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:51.463656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.465065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88510f3-a3ed-4ea4-a486-539484b726d6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.480342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 433/855] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:51.565221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114df5af-4fbe-4845-aef2-43d6de95c8dc None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.605616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.606258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.606537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.606810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.614318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419fe27-a210-406b-9cc1-dc50dbb4462c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.614604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419fe27-a210-406b-9cc1-dc50dbb4462c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.614885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419fe27-a210-406b-9cc1-dc50dbb4462c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.616370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 434/856] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:51.616474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.617950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.621312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.621828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.622107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:51.630278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:51.631222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:51.632125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 423/857] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:51.637304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.645238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:51.646425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:51.651423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.718820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 435/858] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:45:51.748122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.758776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0698cc7-67e0-42e5-9fd4-15fd1c9f90f8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.795500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.796098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.796425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.796698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.797394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:51.797696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.799791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.816677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 424/859] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:51.818866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.844699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.867158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69021-ee5d-45d5-bf3a-c77539a2d885 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.889685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:51.912836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.913733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.914021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.914300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.924965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e11a6-2987-4458-8ee3-d35b321bb7b7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:51.928144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:51.933471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:51.934433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:51.935281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 436/860] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:51.942891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:51.964359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:51.964965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:51.965238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:51.965742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:51.966225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=42c1211f644d45bb8979455c4f28284a) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:51.966526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:51.967943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:51.974492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 425/861] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/role_assignments?user.id=42c1211f644d45bb8979455c4f28284a&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:52.021438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:52.073243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f34064-3776-4b42-82e0-eebbda1af18f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:52.089506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:52.118668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:52.119582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:52.119879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:52.120158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:52.120868 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:52.121518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:52.123727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:52.138605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 437/862] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:45:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:52.140275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:52.170180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ade86-5bbf-4f2c-bc3d-3c22720ac06b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:52.204269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:52.213325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:52.216217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:52.216217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:52.216217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:52.216217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:52.216541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:52.268985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7eb91-786d-4aa2-b172-c4f1ffd2b5df None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:52.295839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 426/863] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:45:52 2022] POST /identity/v3/services => generated 251 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:52.316231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:52.316922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:52.317197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:52.317466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:52.331862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:52.336672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:52.350439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:52.351617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:52.352845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:52.352845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:52.352845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:52.352845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 438/864] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:52.359583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:52.478049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ecb1d-52d8-4999-b7d5-755a4db1e955 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:52.520343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:52.521347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:52.521831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:52.522303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:52.523199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:52.523724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:52.525361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:52.543789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 427/865] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:52.545622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:52.574170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:52.655451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f7ae3-dae6-41de-b791-5af0ccd57913 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:52.693525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:52.694552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:52.695059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:52.695539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:52.703861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:52.707209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:52.719528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:52.720546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:52.722606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:52.728554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 439/866] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:45:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:52.731367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:52.787842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:52.836018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f63bef-dc4e-4490-8c05-60f39b8843a6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:52.874338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:52.874950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:52.875231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:52.875505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:52.876239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:52.876579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:52.879058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:52.902173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 428/867] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 1 17:45:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:52.903591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:52.995192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:53.013309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea2794-8d31-4426-9ad8-912474fc94eb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:53.049755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:53.050474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.050762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.051037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.121189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:53.122634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:53.208750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:53.231888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 440/868] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:52 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:45:53.242024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:53.352913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efde71b-017d-4f0d-84a3-5cc7ec53ac4d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:53.373876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:53.400883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:53.401762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.402438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.403231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.414331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:53.415927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-00d9ac45-8c43-4f91-b21b-6e9375d65d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:53.431618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:53.432841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:53.433967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:53.433967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:53.433967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:53.435658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 429/869] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:53.441753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:53.554991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e662-fc52-4f45-8dbc-d78cd1c1594b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:53.591815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:53.592520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.592799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.593122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.593774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:53.594076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:53.595472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:53.607406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 441/870] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:53.609197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c0e15-3288-48a4-90c1-6571a4728309 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.609690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c0e15-3288-48a4-90c1-6571a4728309 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.610192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c0e15-3288-48a4-90c1-6571a4728309 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.612239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 430/871] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:53.613311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:53.617268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.617548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.617819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.643249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:53.644649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:53.664871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:53.729413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408106b-e0bf-4218-837b-d37ddf846c91 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:53.738306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 431/872] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:45:53.767686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:53.768705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.769000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.769336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.776476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:53.777912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:53.790191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:53.791107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:53.792041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:53.796059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 442/873] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:53.797478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:53.896881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:53.906544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd6c67-6160-4b50-9235-1c41449e1d2b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:53.944843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:53.945457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:53.945738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:53.946013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:53.946720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:53.947030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:53.948543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:53.966805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 432/874] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:53.968370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.071024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.080054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cc63e-4a96-4328-8511-5cf6bb430b26 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:54.081925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:54.121229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:54.121914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.122183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.122379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.123108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=16dc787e53484e0ea5ab6757d99c8a83) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:54.123410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:54.124869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd2c68-4e00-4c47-add7-9b4de8f43789 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:54.131542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 443/875] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:53 2022] GET /identity/v3/role_assignments?user.id=16dc787e53484e0ea5ab6757d99c8a83&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:54.132774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31167e2c-f898-4791-86a0-359c5832c21c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.133063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31167e2c-f898-4791-86a0-359c5832c21c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.133340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31167e2c-f898-4791-86a0-359c5832c21c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.134704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 444/876] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:54.142590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56310481-088c-44cd-a10d-e63e5e1cbdd8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.143392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56310481-088c-44cd-a10d-e63e5e1cbdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.144465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56310481-088c-44cd-a10d-e63e5e1cbdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.172548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-56310481-088c-44cd-a10d-e63e5e1cbdd8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:54.175320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-56310481-088c-44cd-a10d-e63e5e1cbdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:54.208644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876830-4ed8-4f4c-a47f-bd21d91a9246 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:54.264249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:54.265154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.265856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.266272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.268568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:54.271019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:54.284882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 445/877] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:54 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:54.295675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c5532-72b7-432a-90b6-36ac130af25d None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.295987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c5532-72b7-432a-90b6-36ac130af25d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.296267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c5532-72b7-432a-90b6-36ac130af25d None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.297884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 446/878] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:45:54.302730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.303059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.303337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.322073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:54.323254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:54.397789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 433/879] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:45:54 2022] POST /identity/v3/services => generated 269 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:54.404728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 447/880] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:45:54.558304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.575231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:54.670765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e8896-720b-41a7-a421-660367a31de8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:54.703298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.706254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:54.706887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.707162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.707429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.708011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:54.709421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:54.718840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:54.719889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:54.722625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 434/881] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:45:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:54.726752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.730018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:54.810084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf0f1c-9e79-4bf9-a7f8-cf7d5af6dddf None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:54.824674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:54.834717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a38840-1ef1-4676-aa9a-dd52123e13e9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:54.846580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:54.847229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.847592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.847873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.848491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:54.849843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:54.858857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:54.861451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:54.864402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:54.871204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 448/882] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:54.871204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.871487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:54.876354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:54.876354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:54.876354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:54.876354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:54.876354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:54.876354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:54.880569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 435/883] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:45:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:54.886422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:54.997464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:55.016533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6768bee-7049-4a8d-8659-8fbb3a819e79 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:55.016713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d0f8-5df0-437d-85dd-f2543c2ae98b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:55.055924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:55.056887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:55.057318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:55.057742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:55.058726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:55.059100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:55.059264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:55.059761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:55.060045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:55.060388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:55.060801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:55.060976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:55.061283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:55.063273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:55.068645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 436/884] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:45:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:55.070904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 449/885] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:55.071566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:55.075447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:55.185904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa15a9-7989-4a9d-bad1-3307c9b92217 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:55.222992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:55.223600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:55.223881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:55.224157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:55.224836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:55.225140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:55.226500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:55.231305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 437/886] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:55.238252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:55.351284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ef0b8-de37-4c9c-b3e0-bd7af3c8d570 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:55.388430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:55.389049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:55.389331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:55.389599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:55.390313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] RBAC: Adding query filter params (interface=public, service_id=8ba630cc9cca4a0e922204d6bf6c06c5, region_id=RegionOne) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:55.390628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:55.391976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5b7d-9eb0-4669-ab99-30dfb47cf35e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:55.398231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 450/887] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:45:55 2022] GET /identity/v3/endpoints?service_id=8ba630cc9cca4a0e922204d6bf6c06c5&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:55.404459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:55.477283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:55.513075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d6808-153f-4b58-8ab4-381da7ccee27 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:55.549916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465a8b5-4418-469b-b1b9-fb65ebb72e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:55.550943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465a8b5-4418-469b-b1b9-fb65ebb72e35 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:55.551426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465a8b5-4418-469b-b1b9-fb65ebb72e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:55.551888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465a8b5-4418-469b-b1b9-fb65ebb72e35 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:55.552716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465a8b5-4418-469b-b1b9-fb65ebb72e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:55.554286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465a8b5-4418-469b-b1b9-fb65ebb72e35 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:55.560970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 438/888] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:45:55 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:56.275943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a45be-0079-4435-9995-79f8197300b2 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:56.276438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a45be-0079-4435-9995-79f8197300b2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:56.276516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a45be-0079-4435-9995-79f8197300b2 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:56.281263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 451/889] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:56.283165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe4927-b3cd-4fd1-b374-69f9cab89ebb None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:56.283714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe4927-b3cd-4fd1-b374-69f9cab89ebb None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:56.283948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe4927-b3cd-4fd1-b374-69f9cab89ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:56.302894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-28fe4927-b3cd-4fd1-b374-69f9cab89ebb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:56.304125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-28fe4927-b3cd-4fd1-b374-69f9cab89ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:56.374428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 439/890] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:56.495074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207745c-c776-4f9d-99e8-efb18101b8cf None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:56.495424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207745c-c776-4f9d-99e8-efb18101b8cf None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:56.495624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207745c-c776-4f9d-99e8-efb18101b8cf None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:56.497514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 452/891] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:56.503159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:56.503775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:56.504347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:56.526515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:56.528029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:56.592257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:56.604896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 440/892] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:56.645513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207745c-c776-4f9d-99e8-efb18101b8cf None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:56.662482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:56.756416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207745c-c776-4f9d-99e8-efb18101b8cf None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:56.792920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:56.793557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:56.793839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:56.794391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:56.805173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:45:56.810155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:56.811083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:56.811951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 453/893] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 1 17:45:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:56.819081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:56.894315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:56.927478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a970329-92c1-47b4-802a-58b26deef1a8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:56.966318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:56.967295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:56.967721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:56.968222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:56.969134 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:56.969586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:56.971474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:56.981006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 441/894] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:45:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:56.986060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:56.987790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.009437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47209b-d154-4753-b851-1e7ce6536b21 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.047294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.047936 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.048219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.048523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.049191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.050822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.051775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:57.061061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:57.070860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:57.071772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 454/895] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 1 17:45:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:57.072613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.109998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e558e0b-3ffe-4ed3-b6ca-ced6304e72e2 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.152879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.153512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.153792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.154066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.165625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.167488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-5ee5b2e9-1fa0-446d-aa6d-b62d9f60c579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:57.182891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:57.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:57.184846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:57.184846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:57.184846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:57.184846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 442/896] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:57.185516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf23e-d5bf-4296-81f1-18c3daaf134e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.185516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf23e-d5bf-4296-81f1-18c3daaf134e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.185763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf23e-d5bf-4296-81f1-18c3daaf134e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.187083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 443/897] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:57.192340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf23e-d5bf-4296-81f1-18c3daaf134e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.197805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31015a26-baf4-4bf3-a4b6-39a9582cf475 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.237039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.238212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.238489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.238759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.239393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:57.239691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.241513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bcb91-4dae-4687-a2b4-1a70fb8e1eb5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.246373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 455/898] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:57.253182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.253467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.253747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.260894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.279818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:57.281041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:57.308839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf23e-d5bf-4296-81f1-18c3daaf134e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.317156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.348412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.349398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.349681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.349955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.350662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:57.350972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.360607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 456/899] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:57.360751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.360860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.376288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 444/900] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:57.382302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.496678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b377a-d7be-41b2-a406-8126ede2c75b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.507072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21440b89-c6d2-4e02-8f3d-46496d7a1ed6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.528767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.534504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.535273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.535617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.535953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.544801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:57.544801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.544801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad369c-cb75-4ec5-a0b4-d0226382e871 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.544801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 457/901] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:57.544801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207401-8e79-4a5e-9629-14fc535ac20e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.545333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207401-8e79-4a5e-9629-14fc535ac20e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.545333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207401-8e79-4a5e-9629-14fc535ac20e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.546397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 458/902] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:45:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:45:57.548954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207401-8e79-4a5e-9629-14fc535ac20e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.550793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.551426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.551702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.551974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.559966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.561422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-b7fb5cae-501b-4eee-b796-e59bb39bc910 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:45:57.576208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:57.577352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:57.578331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 445/903] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:57.581494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912df291-0a06-42be-a82c-605739f79360 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.581788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912df291-0a06-42be-a82c-605739f79360 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.582059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912df291-0a06-42be-a82c-605739f79360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.601179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-912df291-0a06-42be-a82c-605739f79360 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:45:57.602344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-912df291-0a06-42be-a82c-605739f79360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:45:57.623151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.664401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207401-8e79-4a5e-9629-14fc535ac20e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.673683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 446/904] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:45:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:45:57.674762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912df291-0a06-42be-a82c-605739f79360 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.702062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.702728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.703011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.703312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.704048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] RBAC: Adding query filter params (interface=public, service_id=9c48be20c9254fa69de46ad8971c477b, region_id=RegionOne) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:57.704383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.706929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.712785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 459/905] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/endpoints?service_id=9c48be20c9254fa69de46ad8971c477b&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:57.713731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.781187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.790738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912df291-0a06-42be-a82c-605739f79360 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.824748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cd7f4-8ff9-4f85-9601-801ae53e4400 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:57.826602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.827216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.827493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.827764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.828521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:57.828828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.830800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.853150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 447/906] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:57.853150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.868231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:57.873332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:57.874156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:57.874428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:57.874694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:57.875348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:57.876737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:57.972762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 460/907] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:45:57 2022] POST /identity/v3/services => generated 241 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:57.973596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:57.974819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740d7be-8814-4faf-9261-3f852ac1956b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.025470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.026096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.026380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.026648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.027269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.028721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.044436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 448/908] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:45:58.044436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.116676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee92531-e21d-467f-a9cb-410f3bc16797 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.144484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.158866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.166901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.167539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.167826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.168098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.196797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f3d0c-4b5e-4f09-a63f-6a3e350ef5da None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.248350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.255966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.256251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.256550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.257436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.260491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.271476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.273823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:58.276212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:58.281212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:45:58.282077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 449/909] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:58.287353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.390839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 461/910] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:45:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:45:58.393155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.398627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fd733-d35a-4116-8de5-9cd75c3b3271 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.415686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.435826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.436616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.436998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.437373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.438395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:58.438395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.440088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.445703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 450/911] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:45:58.453584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.458587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.520959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d22f2-e515-4add-8236-9d79aa30b48f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.567176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.568202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.568787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.569229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.572528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32732a8-832a-4fd5-b85b-8173c55c1673 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.582795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.585175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:45:58.605462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:58.606666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:45:58.607805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:45:58.607805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:45:58.607805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:58.609867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 462/912] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:45:58.614407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.615173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.616112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.616780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.617052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.617689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:58.617990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.619334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.625641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 451/913] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 1 17:45:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:58.634152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.694654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.715845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.738996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80025ab-4a1d-45fe-a012-3c406a0be804 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.755114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aa66e-14a9-4d1a-a1af-e6bfc1b617c1 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.786748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.787865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.788382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.789064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.790547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:58.791099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.792927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.794846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.795874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.795996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.796384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.797145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.798677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:58.812955 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 463/914] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:58.814481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:58.922149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a6da-ec97-4515-be5d-8a6e0e7c72cf None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:58.924778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 452/915] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:45:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:45:58.964040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:58.974183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:58.975263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:58.975750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:58.976223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:58.984786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:58.986678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:45:59.000794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:45:59.001778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:45:59.003211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:45:59.005873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 464/916] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:45:59.012005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:59.128047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d3a38-9a46-4352-895b-eb22a07fef26 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:59.154261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:45:59.169231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:59.169876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:59.170160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:59.170436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:59.171162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:59.171471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:59.172953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d785-d8b0-4642-ba47-a02dc03722b6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:59.186884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 453/917] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:45:59.193815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:45:59.310911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7601d-49ce-42c0-b2fa-e4af9c29ed88 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:45:59.349258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:45:59.349874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:45:59.350156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:45:59.350431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:45:59.351160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] RBAC: Adding query filter params (role.id=e5bbb3292b7e4e49b38b635b88aa487d, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=16dc787e53484e0ea5ab6757d99c8a83) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:45:59.351468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:45:59.353092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffeb5eb-c091-4835-83f8-5a0de6c7ce12 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:45:59.358958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 465/918] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:45:59 2022] GET /identity/v3/role_assignments?user.id=16dc787e53484e0ea5ab6757d99c8a83&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=e5bbb3292b7e4e49b38b635b88aa487d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:00.030695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc6585-1ccd-40a2-9ce7-188eba843e99 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.031240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc6585-1ccd-40a2-9ce7-188eba843e99 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.031240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc6585-1ccd-40a2-9ce7-188eba843e99 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.032789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 454/919] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:00.035723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752e8ec-71b7-4af6-a100-1c2dca4882e1 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.036105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752e8ec-71b7-4af6-a100-1c2dca4882e1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.036281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752e8ec-71b7-4af6-a100-1c2dca4882e1 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.037933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 466/920] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:00.039251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.039537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.039810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.045733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.046016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.046289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.063063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:00.064263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:00.065578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:00.066932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:00.127676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:00.153357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 455/921] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:00.159090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 467/922] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:00.427974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.440145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.557767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e369336-3e73-47e9-8d3c-a996b68826fd None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:00.558691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36736dfd-a723-4d67-96fc-038b6e1ddbbc None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:00.576005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:00.594410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:00.595332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.596431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.596867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.597609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:00.601858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:00.603892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:00.604637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.604988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.605332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.605992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:00.607419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:00.610083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:00.611065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:00.611932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:00.619442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:00.620412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 17:46:00.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:00.622565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 468/924] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:46:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:00.623854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 456/924] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 1 17:46:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:00.625013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.625460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.678998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:00.745131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f9253-92c9-46fe-9ac3-d6c5aebffe7b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:00.756840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdeeb3-ea4b-4ede-89e0-be08f7af7cf5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:00.789261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:00.789880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.790454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.790734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.791377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:00.791682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:00.793281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:00.794568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:00.795440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.795922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.796423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.797267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:00.797771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:00.799494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:00.803432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 457/925] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:46:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:00.805106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.812419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 469/926] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:00.813579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.856243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:00.928193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d04d1-5290-4b2a-a073-8358c3ff30ff None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:00.941024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9eb3-bfb2-45c6-83c1-b9c69079eb4a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:00.953062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:00.965788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:00.966401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.966698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.966954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.967591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:00.967889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:00.969275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeff94-0d89-40da-882f-94b99b12d64a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:00.974291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 458/927] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:46:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:00.975449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662486-5083-4df7-9c93-d0d2c968861a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.975743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662486-5083-4df7-9c93-d0d2c968861a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.975995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662486-5083-4df7-9c93-d0d2c968861a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.980477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 459/928] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:00.981484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662486-5083-4df7-9c93-d0d2c968861a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:00.983802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:00.984561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:00.984911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:00.985211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:00.986242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:00.986764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:00.988858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748b927-9d8a-416b-97c2-573733745a35 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:00.994273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 470/929] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:01.000091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.000398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.000672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.020501 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:01.021689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:01.096343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 471/930] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:01.096343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:01.099555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662486-5083-4df7-9c93-d0d2c968861a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:01.102207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:01.147738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bc115-efbb-4701-be40-e4ddba217bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:01.149147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bc115-efbb-4701-be40-e4ddba217bdf None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.149844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bc115-efbb-4701-be40-e4ddba217bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.150273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bc115-efbb-4701-be40-e4ddba217bdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.151345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bc115-efbb-4701-be40-e4ddba217bdf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:01.153265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bc115-efbb-4701-be40-e4ddba217bdf None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:01.211677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca12e-b2e6-487a-8eee-d159c0ef20fe None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:01.214825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:01.256941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:01.257572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.257849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.258119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.258831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] RBAC: Adding query filter params (interface=public, service_id=196bcd71840743658a0ffc8895b55314, region_id=RegionOne) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:01.259142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:01.260692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:01.268438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 472/931] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:46:01 2022] GET /identity/v3/endpoints?service_id=196bcd71840743658a0ffc8895b55314&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:01.272565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:01.281980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 460/932] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:46:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:01.354025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84de1bc-b57a-4368-ae88-e1716d6730bf None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.354371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84de1bc-b57a-4368-ae88-e1716d6730bf None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.354589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84de1bc-b57a-4368-ae88-e1716d6730bf None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.357735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 461/933] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:01.364648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.364991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.365271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.386376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:01.390455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:01.394731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ec7a-947c-431c-aeda-3a21bc2c6c71 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:01.441586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:01.442591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.442591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.442877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.443486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:01.445064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:01.453021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 473/934] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:46:01 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:46:01.453817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:01.465984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 462/935] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:01.492570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:01.577958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f27d6-2109-4abd-92c9-6eb3fc07f2e5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:01.618989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:01.619612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.619893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.620167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.620953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:01.622377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:01.625578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING keystone.server.flask.common [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 17:46:01.672545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.sql.core [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:46:01.672545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 17:46:01.672545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [parameters: {'id': 'b94c55b875e04912817e4a459be09d07', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 17:46:01.672545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81704) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b94c55b875e04912817e4a459be09d07.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b94c55b875e04912817e4a459be09d07. Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:46:01.691753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application result.read() Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 17:46:01.696950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application yield resource Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise self.value Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application session.commit() Sep 01 17:46:01.697974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:46:01.698907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:46:01.699970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application result.read() Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:46:01.703416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [parameters: {'id': 'b94c55b875e04912817e4a459be09d07', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:01.705792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:01.707687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b94c55b875e04912817e4a459be09d07. Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.713516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 474/936] 10.222.0.15 () {60 vars in 1140 bytes} [Thu Sep 1 17:46:01 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:01.714064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:01.740571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:01.820646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:01.833635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:01.835319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315c8e-ca7d-43be-99d8-0af43171ac7d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:01.851547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c030b2-0453-4295-8513-a63fcf975a3b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:01.879948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:01.881158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.881580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.882117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.887449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:01.888217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:01.888595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:01.888937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:01.890516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:01.893561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:01.902150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:01.907364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:01.909077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:01.910004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:46:01.911204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:01.912047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:01.913019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:01.919499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 475/938] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 1 17:46:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:01.919499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:01.920061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 463/938] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:01.922055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:02.042333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0935-d0e5-4004-a480-f9b3403f3e63 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:02.047943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293bd3b-149b-4591-bd51-208b8fad7e54 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:02.079358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:02.080663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:02.080947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:02.081218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:02.081998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:02.082276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:02.083702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:02.091178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:02.092480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:02.092954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:02.093401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:02.094246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:02.095164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:02.097634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:02.107202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 464/939] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:02.122769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 476/940] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:46:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:02.124425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:02.138527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:02.238117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768e9ef-31fe-4749-825a-fe49a541c9c3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:02.280873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:02.281520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:02.286332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:02.286332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:02.293366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:02.294834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:02.310721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:02.315820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:02.316904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:02.316904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:02.316904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:02.316904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 465/941] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:02.318990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:02.330829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:02.430383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486a350-fcc7-4c22-a788-b8aacbd78796 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:02.466436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:02.467056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:02.467333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:02.467603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:02.468253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:02.468586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:02.469939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:02.486400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 477/942] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:02.488625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:02.566089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:02.605052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146acf-03f2-40e8-991f-33008a1a6ba5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:02.655103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:02.656135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:02.657149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:02.657659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:02.666592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:02.669658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:46:02.690529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:02.691554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:46:02.692540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:02.695199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 466/943] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:46:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:02.700943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:02.776483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:02.817149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76b31a-0ab4-4c8c-9e1e-ef48331cc120 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:02.854395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:02.855411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:02.855926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:02.856466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:02.857420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:02.857924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:02.859531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:02.880584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 478/944] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 1 17:46:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:02.889707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:03.015504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899c721-fee4-4def-9686-af8393d746f6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:03.044698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:03.054125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:03.054777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.055385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.055651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.070119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:03.070119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:03.084094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:03.085891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:03.087599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:03.087599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:03.087599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:03.087599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 467/945] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:03.091478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:03.207287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5647c-f776-44e6-af8d-a83c78094eaa None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:03.233929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:03.246504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:03.247128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.247407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.247681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.255434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:03.255434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:03.255434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:03.262649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 479/946] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:03.270219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:03.387634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e1da2-8e23-41ae-a05b-dbcf92738891 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:03.427002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:03.427945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.428429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.428914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.436627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:03.438343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-ee8f3a8d-fdaf-43a0-bbc6-d30be3935712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:46:03.451782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:03.452796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:03.453727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:03.456049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 468/947] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:03.463105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:03.573724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb28ec1-6429-4ae4-918d-2257bbbb0a93 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.574069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb28ec1-6429-4ae4-918d-2257bbbb0a93 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.574273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb28ec1-6429-4ae4-918d-2257bbbb0a93 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.575910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 469/948] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:03.581267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284b1e-df52-4194-b6b7-b6027bf9c597 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.581575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284b1e-df52-4194-b6b7-b6027bf9c597 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.582271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284b1e-df52-4194-b6b7-b6027bf9c597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.592894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878c7f7-d2c4-4ffd-8701-ab9b2ca0d4ef None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:03.607019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6c284b1e-df52-4194-b6b7-b6027bf9c597 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:03.608638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6c284b1e-df52-4194-b6b7-b6027bf9c597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:03.637234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:03.637902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.638129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.638401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.639096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:03.639404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:03.640877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0354e75-d2dd-4fd6-bc28-945a3cb86fe2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:03.645344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:03.654802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 480/949] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:03.656133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd07552-4063-4bcf-b3f4-9eec53f6c939 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.656626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd07552-4063-4bcf-b3f4-9eec53f6c939 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.656914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd07552-4063-4bcf-b3f4-9eec53f6c939 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.659022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 481/950] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:03.664715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd07552-4063-4bcf-b3f4-9eec53f6c939 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:03.699920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 470/951] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:03.704293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.704293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.704293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.719364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:03.721114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:03.722782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:03.783847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd07552-4063-4bcf-b3f4-9eec53f6c939 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:03.824415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 471/952] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:03.825673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:03.826337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:03.826614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:03.826883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/16dc787e53484e0ea5ab6757d99c8a83/roles/e5bbb3292b7e4e49b38b635b88aa487d` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:03.829359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=16dc787e53484e0ea5ab6757d99c8a83, role_id=e5bbb3292b7e4e49b38b635b88aa487d)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:03.830847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:03.911052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 482/953] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:46:03 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/16dc787e53484e0ea5ab6757d99c8a83/roles/e5bbb3292b7e4e49b38b635b88aa487d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:03.976561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.093031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:04.101753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4e28f-7ab6-4858-9c4b-8ebd81b54362 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.107645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.148157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.149126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.149666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.150354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.152104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.153721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:04.167694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:04.169000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:04.169942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 472/954] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:04.174943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.240810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a78c6-a050-4f58-9a8d-ee426b41d224 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.244804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:04.286720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.288255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.288822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.289575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.290752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.292816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.296330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb76125-18a0-49d5-9517-e3bbbcab83f0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:04.305031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:04.306065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:04.306922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:04.312919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 483/955] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:46:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:04.321166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.342307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.343126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.343407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.343679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.344369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:04.344659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.346770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.357116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 473/956] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:04.358101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.388592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:04.449306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d36d-3673-4427-92b3-df1fb35618c0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.485655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.486200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5fdaa-b221-44ed-90cd-fdf6606fe8f3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.486326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.486571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.486840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.487472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:04.487770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.489263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.499366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 484/957] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:04.500356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.530291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.531325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.532075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.532785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.534002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:04.534460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.534949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:04.536840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b774ee-1043-43a5-ba9b-55bb0037840c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.546789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 474/958] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:04.574220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278f62-8cac-4ba7-a80a-0646e2bd6dc4 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.575215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278f62-8cac-4ba7-a80a-0646e2bd6dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.575925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278f62-8cac-4ba7-a80a-0646e2bd6dc4 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.577826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 475/959] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:04.584152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.584805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.585304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.605734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:04.607312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:04.622888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8252b4a-533d-4cec-af69-70b29b930ab8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.663790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.664451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.664735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.665017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.665672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:04.665968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.667349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.673284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 485/960] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:04.678611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:04.703965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 476/961] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:04.797592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:04.805082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceb2c4-3a3a-458f-8ccf-a378ae40e9e9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:04.840673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:04.841283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:04.841566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:04.841892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:04.842622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:04.844036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:04.922698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 486/962] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:46:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:46:04.987876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:05.013989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:05.154863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad221e-1f49-41b1-b2f0-f01d1ffa2765 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:05.199532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:05.200150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:05.200462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:05.200740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:05.201394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:05.202834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:05.205431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING keystone.server.flask.common [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 17:46:05.277877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.sql.core [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:46:05.277877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 17:46:05.277877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [parameters: {'user_id': 'e3bb1fc8a103494389248c18976a5573', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 17:46:05.277877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81703) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result.read() Sep 01 17:46:05.286043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:46:05.290508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application yield resource Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application next(self.gen) Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise self.value Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application session.commit() Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 17:46:05.291483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.session.flush() Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application compat.raise_( Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 17:46:05.292719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 17:46:05.294195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application conn.query(q) Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application result.read() Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 17:46:05.295968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e3bb1fc8a103494389248c18976a5573', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:05.298366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 17:46:05.305587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:05.307817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 477/963] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 1 17:46:04 2022] POST /identity/v3/users => generated 160 bytes in 301 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:05.308774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:05.406609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffb922-aaf2-4a7f-8193-d693833ccb11 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:05.446103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:05.446750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:05.450914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:05.450914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:05.457620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:05.460344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f89aad2e-e020-40b3-be06-957c84258c69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:46:05.474929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:05.477171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:05.479398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 487/964] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:05.487288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:05.572821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:05.598651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4af8d-0e17-4c68-8e42-40b74cb5ff9d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:05.635488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:05.636103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:05.636428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:05.636707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:05.637420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:05.637723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:05.639463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207fde0-ac25-46b9-a33c-801a23c8d473 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:05.679354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 478/965] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 1 17:46:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:06.067806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445099-6445-4e0e-9da3-bd2f27e4f01a None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.068129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445099-6445-4e0e-9da3-bd2f27e4f01a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.068402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445099-6445-4e0e-9da3-bd2f27e4f01a None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.069895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 488/966] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:06.074966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.075261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.075539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.096368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:06.097994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:06.170132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 479/967] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:06.446538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445099-6445-4e0e-9da3-bd2f27e4f01a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:06.462706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:06.552371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445099-6445-4e0e-9da3-bd2f27e4f01a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:06.587570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e8a65-33fe-49c6-b88e-80ee42130235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:06.588198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e8a65-33fe-49c6-b88e-80ee42130235 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.588508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e8a65-33fe-49c6-b88e-80ee42130235 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.588782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e8a65-33fe-49c6-b88e-80ee42130235 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.599803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e8a65-33fe-49c6-b88e-80ee42130235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-db8e8a65-33fe-49c6-b88e-80ee42130235 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:06.604686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:06.605618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:06.606499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 489/968] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 1 17:46:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:06.613329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:06.676429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:06.716175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853479-458e-42cc-b4c7-b8181d141441 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:06.716824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef62170-c91f-4f18-9129-219dc80e505a None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.717117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef62170-c91f-4f18-9129-219dc80e505a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.717397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef62170-c91f-4f18-9129-219dc80e505a None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.719483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 490/969] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:06.724690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.724976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.725249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.744331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:06.745869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:06.752422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:06.753034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.753316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.753584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.754254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:06.754555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:06.755911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:06.765743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 480/970] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:46:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:06.771993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:06.819853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 491/971] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:06.877347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe784e-e9fc-4742-8ae7-6a318b824458 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:06.901374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:06.913114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:06.913749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.914030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.915029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.925728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:06.927274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-921081f8-f5fe-4364-ae0d-455e8c19a1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:06.941837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:06.942849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:06.943785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:06.943785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:06.943785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:06.947937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 481/972] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:06.948794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:06.951566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:06.972412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d4731-b44f-4854-9b1b-4df724bc4a22 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.972412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d4731-b44f-4854-9b1b-4df724bc4a22 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.972412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d4731-b44f-4854-9b1b-4df724bc4a22 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.972412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 482/973] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:06.975941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:06.976231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:06.976534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:06.995207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:06.996416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:07.056026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74a075-fcae-4e22-950a-9d0ef728fa67 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.064095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 483/974] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:07.090433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.091399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.091399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.091666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.092360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.092464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:07.092670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.094023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.108021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 492/975] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:07.113570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.177760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.202391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598f696-0b97-4a13-953c-5a463160bb74 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.210957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.218392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df800336-31c8-4852-beba-222e4a5237a5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.244627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.245170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.245451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.245724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.246598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.247728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.257919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.258587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.258864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.259181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.267960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.269594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:46:07.280599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:07.281610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:07.282580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 493/976] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:07.287942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.336337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 484/977] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:46:07 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:46:07.337029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.400482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df1e3f-3442-4f6c-88ac-584f24b43ce7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.427421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.447908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.448932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.449479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.449756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.450647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:07.450966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.453068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.454609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778d51d-557e-4776-8c5f-4fae225a93f6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.476210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 494/978] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:07.482435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.498085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.498736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.499019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.499290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.499946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.501352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:07.511302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:07.512483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:07.513363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 485/979] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:07.518025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.554142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.607373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c943b9-a189-4f3f-8f97-5eb0a8dc0f60 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.633241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d5df-ee9d-4d9a-af4b-6cb146fe0e11 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.644577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.645188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.645465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.645739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.670385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.670936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.671216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.671483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.672128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:07.672457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.674371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.679257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 486/980] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:07.684811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.694687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.720163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.722317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.787968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.794919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93c7e-5488-4fef-8076-a7dc23953034 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.826696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 495/981] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:46:07.832737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.836732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.837344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.837620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.837889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.838526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:07.838826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.840178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:07.846390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 487/982] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:07.852284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:07.911074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:07.935387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306dd67-dbe9-43e4-b865-2e797b047dce None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.963540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a06ef6-a8af-4af0-8789-1cd4f0775485 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:07.974856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:07.975493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:07.975775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:07.976048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:07.986745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:07.988178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.000208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:08.000861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:08.001136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:08.001403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:08.002149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] RBAC: Adding query filter params (interface=admin, service_id=196bcd71840743658a0ffc8895b55314, region_id=RegionOne) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:08.002469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:08.003450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:08.004424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:08.010002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:08.010002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:08.010002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:08.010178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.010251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 496/983] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:08.010251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:08.015089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 488/984] 10.222.0.15 () {56 vars in 1249 bytes} [Thu Sep 1 17:46:07 2022] GET /identity/v3/endpoints?service_id=196bcd71840743658a0ffc8895b55314&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:08.021152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:08.047962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:08.117712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dc3f6-af86-4dc0-aaf7-78ffd2eb08c8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:08.125311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d32b96-c631-426a-8875-48ef54a38483 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:08.144372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:08.152807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:08.153410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:08.153763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:08.153965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:08.159775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:08.159775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:08.159775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.160854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:08.161453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:08.161727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:08.161995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:08.162597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:08.163934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.168427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 497/985] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:08.180339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 489/986] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:46:08 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:46:08.180562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:08.296498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c936b2a-6a6e-405f-afeb-7d3bb24f4f59 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:08.336176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:08.336763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:08.337040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:08.337310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:08.344628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:08.346373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:46:08.358210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:08.359189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:08.360294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 498/987] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:08.365857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:08.455514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:08.471796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac5ec-713e-4083-9cd4-10403e5eac35 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:08.506725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:08.507333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:08.507611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:08.507882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:08.508612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:08.508918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:08.510330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140fe1b-3cb2-49e5-99e3-cf452be3fe11 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.527576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 490/988] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:08.529060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:08.578070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:08.630939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8fe68-8058-45f3-9f60-1c0a73d06ce6 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:08.664440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:08.665049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:08.665329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:08.665601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:08.666335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] RBAC: Adding query filter params (role.id=e5bbb3292b7e4e49b38b635b88aa487d, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=16dc787e53484e0ea5ab6757d99c8a83) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:08.666655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:08.668046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8a081-4e4d-45c2-b470-d3b722824efd None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:08.680359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 499/989] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:08 2022] GET /identity/v3/role_assignments?user.id=16dc787e53484e0ea5ab6757d99c8a83&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=e5bbb3292b7e4e49b38b635b88aa487d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:09.317209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6cd58-89f5-44ef-9194-350b0eee6a9d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:09.317726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6cd58-89f5-44ef-9194-350b0eee6a9d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:09.317841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6cd58-89f5-44ef-9194-350b0eee6a9d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:09.319139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 491/990] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:09.324445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:09.324743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:09.325026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:09.343644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:09.395918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:09.422667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 500/991] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:09.701025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6cd58-89f5-44ef-9194-350b0eee6a9d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:09.810719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6cd58-89f5-44ef-9194-350b0eee6a9d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:09.846053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:09.846842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:09.847128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:09.847411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:09.848000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:09.849392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:09.858308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:09.859288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:09.860186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:09.862184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 492/992] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:09.865619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:09.909956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:09.967429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db63a7-2d9e-4e70-839c-d8ff1b11fda4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.001108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.002263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.002542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.002812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.003441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:10.003744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.005110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249ef34-d01d-40c8-b165-e09ef71d44e2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.014451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 501/993] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:10.015821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:10.029424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:10.091556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbc72d-3c9a-462d-926c-cdd8c10455a5 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.092197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbc72d-3c9a-462d-926c-cdd8c10455a5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.092697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbc72d-3c9a-462d-926c-cdd8c10455a5 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.094387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 502/994] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:10.099780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.100060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.100358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.120467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:10.121643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:10.124774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6698c04-9ca3-4060-ad41-6a6b3057bab2 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.162253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.162855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.163132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.163401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.164025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:10.164347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.165714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.170805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 493/995] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:10.177154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:10.189624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 503/996] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:10.260818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:10.286878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b557-1d6c-4ac8-8af9-f091df26c8c6 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.321019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.322782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.322782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.322782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.323009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] RBAC: Adding query filter params (interface=public, service_id=41eb176d7fce477f82cd5873635c846f, region_id=RegionOne) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:10.323209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.324513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.330379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 494/997] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:46:10 2022] GET /identity/v3/endpoints?service_id=41eb176d7fce477f82cd5873635c846f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:10.335805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:10.372093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:10.436395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4fe2e-ceba-4f32-b06c-ac68f95fe3c7 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.457870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:10.473326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4d06c-3318-4256-b262-cec74c1d50be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.473946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4d06c-3318-4256-b262-cec74c1d50be None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.474225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4d06c-3318-4256-b262-cec74c1d50be None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.474498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4d06c-3318-4256-b262-cec74c1d50be None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.475109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4d06c-3318-4256-b262-cec74c1d50be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.476524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4d06c-3318-4256-b262-cec74c1d50be None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.482831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 504/998] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:46:10 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:46:10.569817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a55601-3559-45de-bdd0-b26aa93a16cd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.607862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.608905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.609363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.609808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.610576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.613030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:10.622964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:10.624031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:10.624969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 495/999] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 1 17:46:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:10.626305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a500b0-7bec-4727-9db1-09066ba32b39 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.626594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a500b0-7bec-4727-9db1-09066ba32b39 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.626867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a500b0-7bec-4727-9db1-09066ba32b39 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.628208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 505/1000] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:10.630319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:10.633196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.633414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:10.633523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.633741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.654243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:10.655589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:10.732356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 506/1001] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:10.748668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1845e4-a80f-423e-8e47-77b232e83223 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.784298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.784918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.785191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.785457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.786085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:10.786385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.787723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.797583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 496/1002] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:10.798852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:10.829237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:10.900542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743daa-3b10-4969-b900-31ef1c54b55a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:10.935581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:10.936202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:10.936514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:10.936796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:10.937667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:10.938228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:10.940347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:10.947867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 507/1003] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 1 17:46:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:10.953395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:11.007625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:11.022382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:11.061994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4095e-fc58-4a6c-b955-eb206c9c9969 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:11.099437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:11.100088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:11.100406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:11.100681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:11.101356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:11.102861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:11.105440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:11.113240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850d012-76ad-4d19-bdab-b14b5ad02c3c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:11.148055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:11.148714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:11.149009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:11.149265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:11.149851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:11.151190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:11.160708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:11.161901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:11.163261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 508/1004] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 1 17:46:11 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:11.167611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:11.177642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 497/1005] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:46:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:46:11.289552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61aa4-204f-431b-82dd-984ac6adfae5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:11.333736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:11.334373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:11.334647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:11.334917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:11.335543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:11.335840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:11.337251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27898ad-5b38-4773-9bbe-392339d575e3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:11.342135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 509/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:11.348137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:11.454174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa119012-2ab0-4da1-9b32-bc4bc4b8f2f7 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:11.481326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:11.489981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:11.490787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:11.491167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:11.493357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:11.493357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] RBAC: Adding query filter params (type=image) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:11.493357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:11.493984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3d83f-005e-4e69-8602-356826ef53ed None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:11.498730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 498/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:12.431535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062055f6-6be0-459e-b616-602a1d4ae7b2 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:12.432154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062055f6-6be0-459e-b616-602a1d4ae7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:12.432154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062055f6-6be0-459e-b616-602a1d4ae7b2 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:12.433577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 510/1008] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:12.438605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:12.439015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:12.439836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:12.457909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:12.459086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:12.532458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 499/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:12.809140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062055f6-6be0-459e-b616-602a1d4ae7b2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:12.831156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:12.916757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062055f6-6be0-459e-b616-602a1d4ae7b2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:12.950880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:12.951517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:12.951798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:12.952072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:12.952685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:12.954028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:12.963389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:12.964769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 17:46:12.965717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:12.967148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 511/1010] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:12.971424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:13.072807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b4714-0a5d-49c4-a921-ddc721e3e6a9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:13.093230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:13.107866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:13.108648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.109022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.109388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.110120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:13.110550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:13.112075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889a3e6-f03e-42a9-a6df-535bab1636a1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:13.122043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 500/1011] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:13.123091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:13.147498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0d255-8a76-4f51-86bc-039927ad0b08 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.147784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0d255-8a76-4f51-86bc-039927ad0b08 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.148058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0d255-8a76-4f51-86bc-039927ad0b08 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.149688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 501/1012] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:13.154381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.154664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.154936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.173293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:13.174636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:13.209413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:13.228696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e698cab-3407-4aa9-b119-bc3231c5f87b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:13.253594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 502/1013] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:13.268545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:13.269170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.269449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.269730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.270340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:13.270631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:13.271978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb2fcf-b3c6-4fac-942a-e6b87459a135 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:13.281713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 512/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 1 17:46:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:13.302246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:13.386054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393f7f-d5c9-4803-825d-bfce15d06a46 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:13.401883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:13.422407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:13.423202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.423601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.423988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.424786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:13.426239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:13.427423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5c4c-00f7-4611-b7fe-b77a8e69e390 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.427713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5c4c-00f7-4611-b7fe-b77a8e69e390 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.427992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5c4c-00f7-4611-b7fe-b77a8e69e390 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.429525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 513/1015] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:13.434149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.434685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.434685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.453427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:13.454610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:13.512786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 503/1016] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:46:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:13.524762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:13.544713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 514/1017] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:13 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:13.661210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937d54-3c5a-4ca2-b2fa-72362500710a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:13.697428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:13.698148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.698497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.698840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.700026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:13.701558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:13.703390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 504/1018] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:13.709755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:13.784244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:13.813889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49f43-fdcc-430b-b8b6-4e538de4053b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:13.817250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:13.852319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5c43b-bf18-4e13-82fe-3255cfff97e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:13.853059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c43b-bf18-4e13-82fe-3255cfff97e4 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.853397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c43b-bf18-4e13-82fe-3255cfff97e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.853791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c43b-bf18-4e13-82fe-3255cfff97e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.854617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5c43b-bf18-4e13-82fe-3255cfff97e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:13.856100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5c43b-bf18-4e13-82fe-3255cfff97e4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:13.860066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:13.924215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc05f-b8a7-4e53-a60e-3cfef1378624 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:13.960441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 515/1019] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:46:13 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:13.963013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97741751-ea51-44ed-a258-4df348d4767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:13.963013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741751-ea51-44ed-a258-4df348d4767d None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:13.963013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741751-ea51-44ed-a258-4df348d4767d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:13.963013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741751-ea51-44ed-a258-4df348d4767d None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:13.963378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97741751-ea51-44ed-a258-4df348d4767d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:13.963475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97741751-ea51-44ed-a258-4df348d4767d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:14.051686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 505/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:46:13 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:46:15.566790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd468c9d-4085-4880-9ff3-28e9a0402ddc None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:15.567313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd468c9d-4085-4880-9ff3-28e9a0402ddc None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:15.567313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd468c9d-4085-4880-9ff3-28e9a0402ddc None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:15.568758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 516/1021] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:15.573717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e78c46-b7e7-41b1-8922-ac5ac203575a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:15.574124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e78c46-b7e7-41b1-8922-ac5ac203575a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:15.574307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e78c46-b7e7-41b1-8922-ac5ac203575a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:15.592768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-50e78c46-b7e7-41b1-8922-ac5ac203575a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:15.593956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-50e78c46-b7e7-41b1-8922-ac5ac203575a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:15.671136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 506/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:15.936040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd468c9d-4085-4880-9ff3-28e9a0402ddc None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:15.938267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91803fe8-e3b1-45b7-8591-921debb6f0f7 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:15.938559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91803fe8-e3b1-45b7-8591-921debb6f0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:15.938826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91803fe8-e3b1-45b7-8591-921debb6f0f7 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:15.940552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 507/1023] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:15.945645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf47ba-fe66-4a4e-8133-c4fe6c41b34c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:15.945931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf47ba-fe66-4a4e-8133-c4fe6c41b34c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:15.946375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf47ba-fe66-4a4e-8133-c4fe6c41b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:15.965786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6adf47ba-fe66-4a4e-8133-c4fe6c41b34c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:15.966971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6adf47ba-fe66-4a4e-8133-c4fe6c41b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:15.985087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.036416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 508/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:16.037661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d6e13-f18c-4439-921c-462d3e9a77a5 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.037949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d6e13-f18c-4439-921c-462d3e9a77a5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.038221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d6e13-f18c-4439-921c-462d3e9a77a5 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.039505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 509/1025] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:16.042808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd468c9d-4085-4880-9ff3-28e9a0402ddc None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.046708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.047221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.047699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.067384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:16.067935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.068660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:16.078705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.082042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.082042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.082042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.092168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:16.097817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:16.098774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:16.100148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 517/1026] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 1 17:46:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:16.104734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.144396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 510/1027] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:16.210337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f841a7-d30f-4af8-ae32-8fb530cafeca None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.246756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.247357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.247629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.247897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.248587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:16.248885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.252130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.259197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 518/1028] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:16.264890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.303735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.354639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.354968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.371389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1488-f473-4cb4-8efe-29c841ff7825 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.406342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.406976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.407256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.407528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.411233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7559c-a1c7-4233-8470-f7a433d9844e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.418221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.419665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:16.435319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:16.436394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:16.437324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:16.437324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:16.437324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:16.441987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 511/1029] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:16.442719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.447196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.447828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.448107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.448417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.449531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.450948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.456694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 519/1030] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:16.456694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.545376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746be329-3d42-40dd-b595-3217c17b4058 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.551569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f534f92-d7bd-4681-b199-38b4cefe3fd3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.565205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.580205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.580941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.581217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.581486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] PATH_INFO: `/v3/services/image` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.582066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.583501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.585237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.585842 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.586133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.586306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.586410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.587027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:16.587327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.588683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:16.592945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:16.594019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:16.594911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 512/1031] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:16.595975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.604160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 520/1032] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:16.604160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.699696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afc775-33fc-4d50-b197-f82d2e146c44 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.701951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f567c-ed33-474f-861e-4f78dfc38792 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.733724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.734322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.734596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.734869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.735489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:16.735600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.735794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.736210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.736617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.736895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.737150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.737542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.738992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.741840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 521/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:16.742605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.783521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.839369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db371c0-2fe8-4f3e-9c53-0630c86717e5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:16.866746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 513/1034] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:46:16 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:16.867574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.873668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:16.874332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:16.875187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:16.875462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:16.885098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:16.886570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:46:16.901626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:16.902718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:16.903763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 522/1035] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:16.908969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:16.941342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:16.990690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3223a-8875-4a2d-a6be-974de11f948f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.020509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:17.025807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf90c-770a-4c45-b36f-78dc8f5287f2 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.034958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.035576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.035858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.036130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.036915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] RBAC: Adding query filter params (type=image) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:17.037275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.038640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.045047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 514/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:17.051702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:17.068935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.069452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.069738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.070014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.070751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:17.071061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.072504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.091827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 523/1037] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:46:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:17.098405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:17.166838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef8c53-516c-4329-876e-a8115c087d0a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.187632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:17.202277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.203013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.203363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.203695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.204498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] RBAC: Adding query filter params (interface=public, service_id=32ebd437f2434470b81b85d3956345a3, region_id=RegionOne) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:17.204974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.206699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.209985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c8cc-d734-4fe2-9f1d-3300ae6bb66c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.216484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 515/1038] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/endpoints?service_id=32ebd437f2434470b81b85d3956345a3&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:17.217517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:17.254339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.255059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.255412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.255708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.280481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:17.324513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.325944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.335237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78bdfa-395d-4362-bf73-6a933f44ac90 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.375230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.376345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.376730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.377067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.378136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.379944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.389019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 516/1039] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 17:46:17.441780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 524/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:46:17.447933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:17.507796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:17.566014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce39c-5caf-493e-ae3c-a2960a6b1e8d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.605368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.606020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.606274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.606545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.617861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.619329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:17.634245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:17.635290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:17.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:17.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:17.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:17.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 517/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:17.641228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:17.701592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:17.747158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab33ca-c3b7-41be-9ce2-60d5d18b93ce None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.781474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.782089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.782371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.782711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.783872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:17.784172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.785551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.803188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 525/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:17.804653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:17.905408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:17.909560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6893410-60e4-4f2d-9eb7-9577ed2f89e9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:17.944045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:17.944714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:17.944995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:17.945269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:17.952668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:17.954120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:46:17.966477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:17.967386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:17.973459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 518/1043] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:17.974010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:18.077132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08124099-7f21-4755-8129-796da2eb2688 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:18.093855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:18.111557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:18.121121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:18.121121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:18.121121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:18.121121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:18.121121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:18.121121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc0993-d692-475e-ad3e-e01ca0e7649d None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:18.132169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 526/1044] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:18.133627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:18.233263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cdf9f-28dc-498e-8f69-47ca8cf17acd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:18.269927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:18.270544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:18.270822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:18.271096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:18.271834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] RBAC: Adding query filter params (role.id=e5bbb3292b7e4e49b38b635b88aa487d, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=99dec0395ef74fd5a6023ae06a1b9538) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:18.272145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:18.273607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156b523-e226-4acd-abc9-dd6db2716788 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:18.284397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 519/1045] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:18 2022] GET /identity/v3/role_assignments?user.id=99dec0395ef74fd5a6023ae06a1b9538&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=e5bbb3292b7e4e49b38b635b88aa487d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:18.866563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05765-8d11-4888-8423-6680b8e36393 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:18.867080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05765-8d11-4888-8423-6680b8e36393 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:18.867080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05765-8d11-4888-8423-6680b8e36393 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:18.868514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 527/1046] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:18.873648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9aa3a-b5c8-4306-a2ce-a1de9a64b46a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:18.874025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9aa3a-b5c8-4306-a2ce-a1de9a64b46a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:18.874200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9aa3a-b5c8-4306-a2ce-a1de9a64b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:18.892604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-35b9aa3a-b5c8-4306-a2ce-a1de9a64b46a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:18.894170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-35b9aa3a-b5c8-4306-a2ce-a1de9a64b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:18.913667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:18.968465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 520/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:19.232938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05765-8d11-4888-8423-6680b8e36393 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.344784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05765-8d11-4888-8423-6680b8e36393 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:19.358335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017dda5-ab42-4847-8bb4-294fdaa20657 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.358624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017dda5-ab42-4847-8bb4-294fdaa20657 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.358902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017dda5-ab42-4847-8bb4-294fdaa20657 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.364509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 521/1048] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:19.365605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.365895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.366173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.380715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:19.381350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.381631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.381903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.385302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:19.386491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:19.393610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:19.398536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:19.399537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:19.400444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 528/1049] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:19.405471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.444374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:19.466696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 522/1050] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:19.521146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6db45-52be-477d-8faf-6331c3ffd736 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:19.555571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:19.556167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.556485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.556755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.557407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:19.557704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:19.559034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:19.572477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 529/1051] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:19.573552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.660634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:19.676594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c79d82-ef99-46db-9046-d5ec84b0058f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:19.712490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:19.713128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.713407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.713677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] PATH_INFO: `/v3/users/af506ba3f95c472187a4e4ba151a28a8` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.726696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] RBAC: Authorizing `identity:get_user(user_id=af506ba3f95c472187a4e4ba151a28a8)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:19.728131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:19.729969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 523/1052] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/users/af506ba3f95c472187a4e4ba151a28a8 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:19.735608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.749234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.840128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:19.844436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3d25-e6af-4055-82a9-398fe63521c8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:19.861989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619175b7-a92f-4885-8a9a-aa66b03a91a0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:19.880786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:19.881971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.882248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.882521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.898424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:19.899207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:19.899490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:19.899761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:19.900780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:19.902298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:19.911392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:19.912544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:19.913477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 524/1053] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:19.918346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:19.927921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:19.959208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:19.960744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:20.031856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f27c9e-41c7-4a4e-80a0-2378f86cf9e8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.074532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:20.076960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.077571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.077846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.078116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.078746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] RBAC: Adding query filter params (name=image) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:20.084484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:20.084627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 530/1054] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:46:20.085958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:20.086060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:20.092535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 525/1055] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:20.094913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:20.168326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:20.194021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd356f3-7526-4b64-aa33-2c17c0ce7afa None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.200509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007348-943f-4d44-9665-e2c20d94338d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.229138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dc53d-5e3d-4432-9d3d-448b248cf301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.229763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dc53d-5e3d-4432-9d3d-448b248cf301 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.230045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dc53d-5e3d-4432-9d3d-448b248cf301 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.230314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dc53d-5e3d-4432-9d3d-448b248cf301 None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.231433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dc53d-5e3d-4432-9d3d-448b248cf301 None admin] RBAC: Authorizing `identity:get_project(project_id=38509c4f499e40b092456d6848543c3c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:20.232872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dc53d-5e3d-4432-9d3d-448b248cf301 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:20.234516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 531/1056] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:20 2022] GET /identity/v3/projects/38509c4f499e40b092456d6848543c3c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:20.235648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b866a8-75ff-47df-9c2b-5c0760060124 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.235969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b866a8-75ff-47df-9c2b-5c0760060124 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.236243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b866a8-75ff-47df-9c2b-5c0760060124 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.239773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 532/1057] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:20.240732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.241069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b866a8-75ff-47df-9c2b-5c0760060124 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:20.241549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.241837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.242120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.242763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] RBAC: Adding query filter params (type=image) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:20.243065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:20.244455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b6dc-8e1d-4673-b70f-4f2f85458227 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:20.250065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 526/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:20.252035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.252968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.253404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.272977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:20.274177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:20.308234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:20.370867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b866a8-75ff-47df-9c2b-5c0760060124 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.377126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 527/1059] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:20.378204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:20.421632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.422547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.422962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.423377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.424548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=38509c4f499e40b092456d6848543c3c, user.id=af506ba3f95c472187a4e4ba151a28a8) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:20.424884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:20.426778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:20.444354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 533/1060] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:20 2022] GET /identity/v3/role_assignments?user.id=af506ba3f95c472187a4e4ba151a28a8&scope.project.id=38509c4f499e40b092456d6848543c3c&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:20.457515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:20.500471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0568ca-27ae-4c25-9a0a-b19157a304ba None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.538736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.539358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.539641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.539917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.540629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:20.542017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:20.631548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 528/1061] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:46:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:20.649426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:20.776105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8eec0f-3075-4923-8bb1-583aff4b6f7e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.808967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:20.821940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.822571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.822844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.823112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.833973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:20.838986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:20.840148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:20.841321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 534/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 1 17:46:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:20.848026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:20.954196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8800e6b-d30a-4b94-b938-97d955f40060 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:20.974825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:20.989977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:20.990846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:20.991125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:20.991397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:20.992059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:20.992439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:20.993805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:21.002664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 529/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:46:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:21.008287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:21.125886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6b61c-efb1-4c57-a20b-08d02e855b4a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:21.174164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:21.175094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:21.175553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:21.175983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:21.187545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:21.189075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:21.203110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:21.204126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:21.205451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:21.205451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:21.205451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:21.208637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 535/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:21.209986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:21.321879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9493ac8-0012-457c-b788-aa073b630d98 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:21.401277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:21.401881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:21.402159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:21.402427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:21.403064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:21.403363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:21.404755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:21.417223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 530/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:21.422961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:21.466313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:21.555511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3db281-80ea-46a4-a356-a5d912318725 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:21.589833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:21.590459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:21.590739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:21.591015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:21.599113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:21.602227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:46:21.613373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:21.614326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:46:21.615307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:21.620756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 536/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:46:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:21.620882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:21.676128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:21.735196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f013c-a1a1-427a-b6ae-cd98a7af236d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:21.770698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:21.771311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:21.771593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:21.771869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:21.772650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:21.772959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:21.774386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:21.798767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 531/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:46:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:21.801263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:21.900853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:21.905193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c427e45-9fda-4afc-9aec-f51dd8268fc3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:21.939354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:21.940059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:21.940359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:21.940639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:21.950602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:21.952021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:21.966637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:21.967598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:21.968552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:21.968552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:21.968552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:21.972641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 537/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:21.974113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:22.075040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:22.078648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801427d-dd1a-4189-b131-7a7a845068e0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:22.113135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:22.114116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.114871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.115141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.115781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:22.116079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:22.117463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:22.133388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 532/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:22.134539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:22.237825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f8b0c-d556-46e7-ac74-e6309d2617f3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:22.276504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:22.277131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.277414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.277685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.285231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:22.286685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-46346063-b353-48db-80f8-32f4a7e4d768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:46:22.297889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:22.298784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:22.305078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 538/1070] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:22.305642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:22.381185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d372368-03b2-48fd-bcda-55abee5cb9d7 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.381517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d372368-03b2-48fd-bcda-55abee5cb9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.381748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d372368-03b2-48fd-bcda-55abee5cb9d7 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.383229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 539/1071] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:22.387971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e8467-90f3-49ef-a65d-761e3557e544 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.388252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e8467-90f3-49ef-a65d-761e3557e544 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.388570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e8467-90f3-49ef-a65d-761e3557e544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.407416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e71e8467-90f3-49ef-a65d-761e3557e544 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:22.409194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e71e8467-90f3-49ef-a65d-761e3557e544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:22.409839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff0e33-c2c5-4f79-80bf-34c66dd593ab None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:22.412055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:22.449531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:22.450141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.450420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.450691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.451390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:22.451695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:22.453160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:22.467066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 533/1072] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:22.473053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:22.483432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 540/1073] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:22.494885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:22.573843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6ecb1-008e-4659-9d81-6ade7d46144e None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:22.607868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:22.608554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.608837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.609110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/99dec0395ef74fd5a6023ae06a1b9538/roles/e5bbb3292b7e4e49b38b635b88aa487d` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.611278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=99dec0395ef74fd5a6023ae06a1b9538, role_id=e5bbb3292b7e4e49b38b635b88aa487d)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:22.612748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:22.625806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d750-a935-4725-aaf5-5aea52f29e80 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.626110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d750-a935-4725-aaf5-5aea52f29e80 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.626919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d750-a935-4725-aaf5-5aea52f29e80 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.628385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 541/1074] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:22.633363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.633587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.633857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.652809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:22.653974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:22.684455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 534/1075] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:46:22 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/99dec0395ef74fd5a6023ae06a1b9538/roles/e5bbb3292b7e4e49b38b635b88aa487d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:46:22.728554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 542/1076] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:22.754497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:22.846947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:22.867078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ebd96-b67c-4c90-84af-1a46ad4c7a3f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:22.902625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:22.903178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:22.903454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:22.903724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:22.913708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:22.919139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:22.920260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:22.921432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:22.925021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 535/1077] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:22.930179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:22.999963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.008748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.033073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adca483-0e8e-4f8b-977c-d6778f641a34 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.074762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.075685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.076158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.076471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.077180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:23.077439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.078790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.092342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 543/1078] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:23.094534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.120522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920f301-d215-41ce-9776-8a11c803b11f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.159043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.159747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.160125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.160527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.171562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.208122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c64f8-9854-4eae-8fd1-3a5b4548c59c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.234275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.238732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.242574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.247427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.248070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.248378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.248657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] PATH_INFO: `/v3/users/af506ba3f95c472187a4e4ba151a28a8` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.252780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] RBAC: Authorizing `identity:get_user(user_id=af506ba3f95c472187a4e4ba151a28a8)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.255481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.256126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 544/1079] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:23 2022] GET /identity/v3/users/af506ba3f95c472187a4e4ba151a28a8 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:23.261823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.352642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 536/1080] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:46:23.358150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.368918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc83d39-96e9-4547-8684-f27b9363afb1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.403701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.405671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.406143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.406419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.406698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.408439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] RBAC: Authorizing `identity:get_project(project_id=38509c4f499e40b092456d6848543c3c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.409926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.411660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 545/1081] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:23 2022] GET /identity/v3/projects/38509c4f499e40b092456d6848543c3c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:23.417276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.465392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45086-b727-4931-97d9-2d77aa776f3d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.488804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.503237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.503872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.504151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.504460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.515459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.517562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.523522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3a543-ebe1-4bd9-bffd-be2ed324f4d7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:23.534380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:23.535404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:23.536343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:23.536343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:23.536343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:23.540446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 537/1082] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:23.541377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:23.564858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54384f62-23ec-45f2-8c6e-1e3161ff0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.565631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54384f62-23ec-45f2-8c6e-1e3161ff0c9b None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.565974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54384f62-23ec-45f2-8c6e-1e3161ff0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.566314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54384f62-23ec-45f2-8c6e-1e3161ff0c9b None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c/users/af506ba3f95c472187a4e4ba151a28a8/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.568249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54384f62-23ec-45f2-8c6e-1e3161ff0c9b None admin] RBAC: Authorizing `identity:create_grant(project_id=38509c4f499e40b092456d6848543c3c, user_id=af506ba3f95c472187a4e4ba151a28a8, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.569740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54384f62-23ec-45f2-8c6e-1e3161ff0c9b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.633240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:23.641725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3c8f6-2fa8-47fa-a352-f2fe7234e088 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:23.676642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:23.677176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:23.677457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:23.677729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:23.678378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:23.678679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:23.680140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e017d2a-b26e-40ac-8fae-567809e70f89 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:23.682961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 546/1083] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:46:23 2022] PUT /identity/v3/projects/38509c4f499e40b092456d6848543c3c/users/af506ba3f95c472187a4e4ba151a28a8/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 17:46:23.705590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 538/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:24.617271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a064f-4104-4f6e-95ef-703a2ae3aa26 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.617768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a064f-4104-4f6e-95ef-703a2ae3aa26 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.617768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a064f-4104-4f6e-95ef-703a2ae3aa26 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.619057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 547/1085] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:24.624221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:24.624641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:24.624800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:24.643814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:24.645012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:24.713664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 539/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:24.986126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a064f-4104-4f6e-95ef-703a2ae3aa26 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.087053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a064f-4104-4f6e-95ef-703a2ae3aa26 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.112929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:25.120381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.121005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.121279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.121551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.132548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:25.137373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:25.138356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:25.139240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 548/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 1 17:46:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:25.144918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.225999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:25.245423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427257a-f4c2-4e4b-aa5f-14ca85976429 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.283367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.283972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.284254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.284550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.285211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:25.285514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:25.286869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:25.294860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 540/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:25.301580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.398967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d1d72-467b-4a8f-acaf-ed7335024f71 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.431898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.432557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.432838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.433110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.443511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:25.444960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:25.458784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:25.459952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:25.460926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:25.460926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:25.460926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:25.460926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 549/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:25.465935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.567118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14639c9-57a8-4144-b786-a338c5e4ff8d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.600066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.600704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.600979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.601245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.601879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:25.602177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:25.603516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293b984-5e16-40e5-9ac2-5eb75e15a721 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:25.615566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 541/1090] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:25.621320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.623484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e2db-694c-4404-80de-3a3b45b56398 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.623768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e2db-694c-4404-80de-3a3b45b56398 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.624049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e2db-694c-4404-80de-3a3b45b56398 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.625381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 542/1091] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:46:25.630225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.630509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.630787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.637810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:25.648825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:25.650013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:25.665994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:25.718723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 543/1092] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 17:46:25.734366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbf5ad-ccb8-4127-9b08-9d57debbaffe None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.768199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.768853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.769134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.769406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.776760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:25.778169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:46:25.789604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:25.790570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:25.791530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 550/1093] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:25.796451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.891040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7c48-7ba0-4567-a545-cc5644e84e9f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:25.924894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:25.925486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:25.925761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:25.926026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:25.926741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:25.927039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:25.928460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:25.930074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:25.945436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 544/1094] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:25.951188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:25.987165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.047649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.050660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bf964-9d18-4e02-ac63-96a6e0746d58 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.084049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.084684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.084957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.085231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.087934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76559ac8-78cd-4b6a-af38-4426096f6c8f None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.121689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.122312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.122591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.122864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.133529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:26.138396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:26.139420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:26.140301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 545/1095] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:26.146294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.151260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.152717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.199516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.246030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cb5d7-2625-480d-9ff6-803e318ec34d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.250337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.254767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 551/1096] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:46:26.260433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.282622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.283228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.283513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.283781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.284486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:26.284990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.286252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.294339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 546/1097] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:26.300008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.358627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb0f2f-a9b7-4433-b42e-6c4f86ecd988 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.391534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.393080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.393452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.393722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.402732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4dc61-ae63-44df-99f4-fb3b7b7e7409 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.403295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.404752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.417810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:26.419105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:26.425060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:46:26.430327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:26.430327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:46:26.430327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:26.430327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 552/1098] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:46:26.430327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.439932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.440589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.440873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.441149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] PATH_INFO: `/v3/users/af506ba3f95c472187a4e4ba151a28a8` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.442284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] RBAC: Authorizing `identity:get_user(user_id=af506ba3f95c472187a4e4ba151a28a8)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.443734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.445610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 547/1099] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/users/af506ba3f95c472187a4e4ba151a28a8 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:26.451158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.477612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.525143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceaef6-09b2-4f77-a7fc-60f177c6d340 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.552465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e55f50-11ba-418b-8205-975bbefdc9b3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.558116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.558724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.559002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.559304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.559916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:26.560218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.561600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.576524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 553/1100] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:26.582417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.591914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.592550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.592822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.593085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.642242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.660257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.661682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.679228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e559b0-2ade-4b01-9131-ba5578185d9a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.692995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.713803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.714426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.714707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.714975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.721561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.722981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:46:26.734457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:26.735394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:46:26.736941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:26.737996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 554/1101] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:26.742090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.761751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 548/1102] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 17:46:26.767482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.841207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406461d-8d7c-4f7d-816b-224fd46db736 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.847568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.866107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a67cee-6ae2-4320-845c-1bec928f3eec None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:26.875542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.876146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.877037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.877304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.877992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:26.878291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.879802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.892048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 555/1103] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:26.898416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.901140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:26.901771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:26.902053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:26.902325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:26.903446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] RBAC: Authorizing `identity:get_project(project_id=38509c4f499e40b092456d6848543c3c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:26.904919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:26.910933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 549/1104] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/projects/38509c4f499e40b092456d6848543c3c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:26.913369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:26.932108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:26.999767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da2853-3a3d-46cb-a64e-e4b42fd56358 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:27.014208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcb43c-daae-4fdb-9528-3b6c894278ac None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:27.035071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:27.035687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:27.035945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:27.036215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:27.036973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] RBAC: Adding query filter params (role.id=e5bbb3292b7e4e49b38b635b88aa487d, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=99dec0395ef74fd5a6023ae06a1b9538) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:27.037279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:27.038688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76e727-0822-4498-acb1-00a5b23f8480 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:27.044565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 556/1105] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/role_assignments?user.id=99dec0395ef74fd5a6023ae06a1b9538&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=e5bbb3292b7e4e49b38b635b88aa487d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:27.050150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:27.050766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:27.051048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:27.051320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:27.052066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=38509c4f499e40b092456d6848543c3c, user.id=af506ba3f95c472187a4e4ba151a28a8) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:27.052408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:27.053909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accaf2ba-a331-478c-ae79-94cb574ea098 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:27.060746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 550/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:26 2022] GET /identity/v3/role_assignments?user.id=af506ba3f95c472187a4e4ba151a28a8&scope.project.id=38509c4f499e40b092456d6848543c3c&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:29.020849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74f55d-8ba5-434e-bf75-68ac184be51e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.021372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74f55d-8ba5-434e-bf75-68ac184be51e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.021372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74f55d-8ba5-434e-bf75-68ac184be51e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.022727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 557/1107] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:29.027867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.028252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.028448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.046159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:29.047335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:29.056834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:29.112829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 551/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:29.382653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74f55d-8ba5-434e-bf75-68ac184be51e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.476245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:29.482903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74f55d-8ba5-434e-bf75-68ac184be51e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.518270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c557a5-5d35-4353-90fb-e5be63ce0b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.518879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c557a5-5d35-4353-90fb-e5be63ce0b8e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.519152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c557a5-5d35-4353-90fb-e5be63ce0b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.519420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c557a5-5d35-4353-90fb-e5be63ce0b8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.520643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c557a5-5d35-4353-90fb-e5be63ce0b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.522047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c557a5-5d35-4353-90fb-e5be63ce0b8e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.523601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 558/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:29.529388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:29.628546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebc798-05b4-491b-b28e-6f655158e6f3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:29.662416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56c92-c970-41d6-8710-0db076d96ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:29.662953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56c92-c970-41d6-8710-0db076d96ae4 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:29.663233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56c92-c970-41d6-8710-0db076d96ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:29.663502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56c92-c970-41d6-8710-0db076d96ae4 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:29.664146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56c92-c970-41d6-8710-0db076d96ae4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:29.665606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56c92-c970-41d6-8710-0db076d96ae4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:29.795958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:29.801174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 552/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:46:29 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:46:31.743809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2daa2-c2b5-490e-a19f-c5d6202212ce None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:31.744394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2daa2-c2b5-490e-a19f-c5d6202212ce None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:31.744394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2daa2-c2b5-490e-a19f-c5d6202212ce None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:31.745715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 559/1111] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:31.750753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:31.751905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:31.751905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:31.771086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:31.772275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:31.837657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 553/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:32.107870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2daa2-c2b5-490e-a19f-c5d6202212ce None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:32.207224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2daa2-c2b5-490e-a19f-c5d6202212ce None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:32.240164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:32.240849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:32.241127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:32.241429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] PATH_INFO: `/v3/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:32.242903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] RBAC: Authorizing `identity:get_role(role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:32.244238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:32.245663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 560/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:32 2022] GET /identity/v3/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:32.252790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:32.350767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd32168-ec98-4961-b1a8-3a644f7d822b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:32.378644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:32.384294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:32.384946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:32.385228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:32.385501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] PATH_INFO: `/v3/users/a6837587f3ca4d36a397288c99cd85f1` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:32.399120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] RBAC: Authorizing `identity:get_user(user_id=a6837587f3ca4d36a397288c99cd85f1)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:32.400590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:32.402194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 554/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:32 2022] GET /identity/v3/users/a6837587f3ca4d36a397288c99cd85f1 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:46:32.407959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:32.439189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:32.505879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1519ba-7328-4341-9e33-4ae6f523adb5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:32.544172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:32.544796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:32.545075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:32.545346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:32.608918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:32.610992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:32.639238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:32.705268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 561/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:46:32.711168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:32.807210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e024f2-2f84-4465-bfe5-e56bb417d34b None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:32.840246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d6255-8573-4907-bdb6-14abfb12701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:32.840885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d6255-8573-4907-bdb6-14abfb12701c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:32.841161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d6255-8573-4907-bdb6-14abfb12701c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:32.841427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d6255-8573-4907-bdb6-14abfb12701c None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:32.842523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d6255-8573-4907-bdb6-14abfb12701c None admin] RBAC: Authorizing `identity:get_project(project_id=38509c4f499e40b092456d6848543c3c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:32.843928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d6255-8573-4907-bdb6-14abfb12701c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:32.845621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 555/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:32 2022] GET /identity/v3/projects/38509c4f499e40b092456d6848543c3c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:32.851528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:32.947965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f711-5133-4997-b190-2d6d56aac08c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:32.975791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:32.980574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:32.981197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:32.981472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:32.981739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:32.982457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] RBAC: Adding query filter params (role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=38509c4f499e40b092456d6848543c3c, user.id=a6837587f3ca4d36a397288c99cd85f1) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:32.982761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:32.984147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73092c8d-3e5f-4272-97f5-0ca8a83508b8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:32.989901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 562/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:32 2022] GET /identity/v3/role_assignments?user.id=a6837587f3ca4d36a397288c99cd85f1&scope.project.id=38509c4f499e40b092456d6848543c3c&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:34.853201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b0f6-30ef-4ef7-b715-0d5bec2ee950 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:34.853724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b0f6-30ef-4ef7-b715-0d5bec2ee950 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:34.853724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b0f6-30ef-4ef7-b715-0d5bec2ee950 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:34.855082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 556/1118] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:34.860844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:34.861298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:34.861400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:34.879779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:34.880994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:34.948948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 563/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:35.220723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b0f6-30ef-4ef7-b715-0d5bec2ee950 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:35.279933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:35.323420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b0f6-30ef-4ef7-b715-0d5bec2ee950 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:35.356340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:35.356960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:35.357239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:35.357508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] PATH_INFO: `/v3/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:35.358893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] RBAC: Authorizing `identity:get_role(role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:35.360242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:35.362130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 557/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:35 2022] GET /identity/v3/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:35.367993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:35.463849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb038c-919e-495e-ae97-77257dd4a3b7 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:35.489572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:35.497980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:35.498596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:35.501537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:35.501537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] PATH_INFO: `/v3/users/a6837587f3ca4d36a397288c99cd85f1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:35.501537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] RBAC: Authorizing `identity:get_user(user_id=a6837587f3ca4d36a397288c99cd85f1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:35.501842 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:35.503322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 564/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:35 2022] GET /identity/v3/users/a6837587f3ca4d36a397288c99cd85f1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:35.509022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:35.619635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112c83e-8a2e-4015-8979-2400c3e36618 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:35.650696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:35.664100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b74258-6d51-4a0b-8f52-9489441adcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:35.664758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b74258-6d51-4a0b-8f52-9489441adcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:35.667356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b74258-6d51-4a0b-8f52-9489441adcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:35.667356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b74258-6d51-4a0b-8f52-9489441adcd2 None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:35.667356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b74258-6d51-4a0b-8f52-9489441adcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=38509c4f499e40b092456d6848543c3c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:35.667811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b74258-6d51-4a0b-8f52-9489441adcd2 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:35.669491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 558/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:35 2022] GET /identity/v3/projects/38509c4f499e40b092456d6848543c3c => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:35.675521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:35.772974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ead91e-a6d2-4152-80d7-2ee1690f5cb8 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:35.805575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e961d4-9469-4509-a55e-9c5aa80c273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:35.806216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e961d4-9469-4509-a55e-9c5aa80c273e None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:35.806493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e961d4-9469-4509-a55e-9c5aa80c273e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:35.806762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e961d4-9469-4509-a55e-9c5aa80c273e None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c/users/a6837587f3ca4d36a397288c99cd85f1/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:35.808861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e961d4-9469-4509-a55e-9c5aa80c273e None admin] RBAC: Authorizing `identity:create_grant(project_id=38509c4f499e40b092456d6848543c3c, user_id=a6837587f3ca4d36a397288c99cd85f1, role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:35.810262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e961d4-9469-4509-a55e-9c5aa80c273e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:35.914325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 565/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:46:35 2022] PUT /identity/v3/projects/38509c4f499e40b092456d6848543c3c/users/a6837587f3ca4d36a397288c99cd85f1/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:37.845970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa43d91-76d3-4c26-ac82-fbfe3a22398d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:37.846497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa43d91-76d3-4c26-ac82-fbfe3a22398d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:37.846497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa43d91-76d3-4c26-ac82-fbfe3a22398d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:37.847866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 559/1124] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:37.853007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:37.853425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:37.853559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:37.871923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:37.873140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:37.921412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:37.939158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 566/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:38.207037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa43d91-76d3-4c26-ac82-fbfe3a22398d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:38.306570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa43d91-76d3-4c26-ac82-fbfe3a22398d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:38.339484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:38.340092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:38.340393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:38.340665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] PATH_INFO: `/v3/roles/0293562a90fb4f1ca0fdd75e98e926f7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:38.342028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] RBAC: Authorizing `identity:get_role(role_id=0293562a90fb4f1ca0fdd75e98e926f7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:38.343371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:38.345236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 560/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:38 2022] GET /identity/v3/roles/0293562a90fb4f1ca0fdd75e98e926f7 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:38.351126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:38.447860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86313-4780-42cc-aae2-d93ec282cffc None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:38.481369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:38.481925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:38.482202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:38.482473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] PATH_INFO: `/v3/users/a6837587f3ca4d36a397288c99cd85f1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:38.483573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] RBAC: Authorizing `identity:get_user(user_id=a6837587f3ca4d36a397288c99cd85f1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:38.485012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:38.487112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 567/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:38 2022] GET /identity/v3/users/a6837587f3ca4d36a397288c99cd85f1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:38.492831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:38.550866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:38.589587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfd85d-579f-4a14-9960-6c832d9d26c8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:38.622390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:38.622976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:38.623252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:38.623520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:38.687889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:38.689458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:38.743763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:38.784391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 561/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:46:38.790246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:38.840897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:38.886270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194b2c9-466b-495d-a710-224171e45d55 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:38.919483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2901c-e350-48b9-ba91-0a4057e10754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:38.920094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2901c-e350-48b9-ba91-0a4057e10754 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:38.920401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2901c-e350-48b9-ba91-0a4057e10754 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:38.920675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2901c-e350-48b9-ba91-0a4057e10754 None admin] PATH_INFO: `/v3/projects/38509c4f499e40b092456d6848543c3c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:38.921774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2901c-e350-48b9-ba91-0a4057e10754 None admin] RBAC: Authorizing `identity:get_project(project_id=38509c4f499e40b092456d6848543c3c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:38.923275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2901c-e350-48b9-ba91-0a4057e10754 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:38.924918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 568/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:38 2022] GET /identity/v3/projects/38509c4f499e40b092456d6848543c3c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:38.930893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:39.027476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fdce0-9ed2-4e98-a8c0-2509d772df84 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:39.060542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:39.061133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:39.061414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:39.061683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:39.062404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] RBAC: Adding query filter params (role.id=0293562a90fb4f1ca0fdd75e98e926f7, scope.project.id=38509c4f499e40b092456d6848543c3c, user.id=a6837587f3ca4d36a397288c99cd85f1) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:39.062710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:39.064718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3506613-ac71-4f86-986f-b32812904a98 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:39.070444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 562/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:38 2022] GET /identity/v3/role_assignments?user.id=a6837587f3ca4d36a397288c99cd85f1&scope.project.id=38509c4f499e40b092456d6848543c3c&role.id=0293562a90fb4f1ca0fdd75e98e926f7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:40.960294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a5d2b-1404-4ac5-a6e4-16fe8f87487b None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:40.960847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a5d2b-1404-4ac5-a6e4-16fe8f87487b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:40.960847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a5d2b-1404-4ac5-a6e4-16fe8f87487b None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:40.962184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 569/1131] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:40.967538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:40.967942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:40.968085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:40.986587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:40.987756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:41.005345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:41.056443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 563/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:41.327097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a5d2b-1404-4ac5-a6e4-16fe8f87487b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:41.427716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a5d2b-1404-4ac5-a6e4-16fe8f87487b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:41.447524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:41.461364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151d82-a9d3-4407-8ad7-2422ab3ca5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:41.461986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151d82-a9d3-4407-8ad7-2422ab3ca5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:41.462269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151d82-a9d3-4407-8ad7-2422ab3ca5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:41.462541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151d82-a9d3-4407-8ad7-2422ab3ca5e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:41.463664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151d82-a9d3-4407-8ad7-2422ab3ca5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:41.465150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151d82-a9d3-4407-8ad7-2422ab3ca5e4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:41.466715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 570/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:41.472765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:41.569681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df157d5-cbfe-4421-98c1-bd31c5c4d8d4 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:41.602165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2dd55-eef0-4226-83cd-e1d62aac74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:41.602753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2dd55-eef0-4226-83cd-e1d62aac74b3 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:41.603029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2dd55-eef0-4226-83cd-e1d62aac74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:41.603298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2dd55-eef0-4226-83cd-e1d62aac74b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:41.603944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2dd55-eef0-4226-83cd-e1d62aac74b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:41.605392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2dd55-eef0-4226-83cd-e1d62aac74b3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:41.721908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 564/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:46:41 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 17:46:43.647157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190355a5-e791-41e5-9190-a699becdb2fd None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:43.647722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190355a5-e791-41e5-9190-a699becdb2fd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:43.647722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190355a5-e791-41e5-9190-a699becdb2fd None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:43.649083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 571/1135] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:43.654259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:43.654696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:43.654797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:43.669065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:43.675345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:43.676572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:43.756272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 565/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:44.024825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190355a5-e791-41e5-9190-a699becdb2fd None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:44.125506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190355a5-e791-41e5-9190-a699becdb2fd None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:44.159800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b5b92-d9d4-4a8b-af4e-10c798d1044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:44.160446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b5b92-d9d4-4a8b-af4e-10c798d1044b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:44.160724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b5b92-d9d4-4a8b-af4e-10c798d1044b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:44.160992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b5b92-d9d4-4a8b-af4e-10c798d1044b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:44.162088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b5b92-d9d4-4a8b-af4e-10c798d1044b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:44.163485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b5b92-d9d4-4a8b-af4e-10c798d1044b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:44.165084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 572/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:46:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:44.170889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:44.268933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8f054-aff3-4c12-8d77-b9d0a4ac237e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:44.302076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa09d0-d6bb-4197-9006-6786a361be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:44.302618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa09d0-d6bb-4197-9006-6786a361be36 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:44.302895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa09d0-d6bb-4197-9006-6786a361be36 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:44.303169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa09d0-d6bb-4197-9006-6786a361be36 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:44.303812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa09d0-d6bb-4197-9006-6786a361be36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:44.305264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa09d0-d6bb-4197-9006-6786a361be36 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:44.314597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:44.420410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 566/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:46:44 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:46:46.360849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c8024-554c-4f91-a11e-a2b452c9a879 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:46.361429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c8024-554c-4f91-a11e-a2b452c9a879 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:46.361429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c8024-554c-4f91-a11e-a2b452c9a879 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:46.362721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 573/1139] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:46.367728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:46.368158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:46.368266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:46.386256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:46.387438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:46.454877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 567/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:46.725721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c8024-554c-4f91-a11e-a2b452c9a879 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:46.784687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:46.827673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c8024-554c-4f91-a11e-a2b452c9a879 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:46.861707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:46.862319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:46.862600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:46.862872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:46.872624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:46.877498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:46.878683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:46.880022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 574/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:46.884700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:46.981441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067013fb-8300-40c2-9c18-e302eee2bcb5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:47.003360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:47.014471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:47.015667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:47.015667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:47.015667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:47.016906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:47.017211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:47.018812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:47.026911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 568/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:47.033411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:47.130076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783171a-eabb-4d78-9282-19651ed35d05 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:47.146432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:47.163548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:47.164157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:47.164463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:47.164733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] PATH_INFO: `/v3/users/e281d5e5e56c4a3e9535cf961d99f731` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:47.177983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] RBAC: Authorizing `identity:get_user(user_id=e281d5e5e56c4a3e9535cf961d99f731)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:47.179385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:47.180984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 575/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:47 2022] GET /identity/v3/users/e281d5e5e56c4a3e9535cf961d99f731 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:47.187128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:47.288641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28ec0d-6857-4cc9-89ea-c45da82d62f1 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:47.322370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:47.322968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:47.323243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:47.323510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:47.373113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:47.388630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:47.392565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:47.486580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 569/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:46:47.492695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:47.589649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573012f-737d-48da-92cc-be4659f019c3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:47.622294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea021e-3ace-41c0-8e6f-7615c32b3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:47.622905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea021e-3ace-41c0-8e6f-7615c32b3e06 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:47.623179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea021e-3ace-41c0-8e6f-7615c32b3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:47.623447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea021e-3ace-41c0-8e6f-7615c32b3e06 None admin] PATH_INFO: `/v3/projects/a6bc80a223a14a04b4dce9b3d206da28` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:47.624565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea021e-3ace-41c0-8e6f-7615c32b3e06 None admin] RBAC: Authorizing `identity:get_project(project_id=a6bc80a223a14a04b4dce9b3d206da28)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:47.626749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea021e-3ace-41c0-8e6f-7615c32b3e06 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:47.628377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 576/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:47 2022] GET /identity/v3/projects/a6bc80a223a14a04b4dce9b3d206da28 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:47.634702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:47.715084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:47.730599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fbbec-4379-4a75-b8ec-3248948750b4 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:47.765794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:47.766519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:47.766798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:47.767072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:47.767796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=a6bc80a223a14a04b4dce9b3d206da28, user.id=e281d5e5e56c4a3e9535cf961d99f731) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:47.768101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:47.769806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69474142-e629-4f01-9789-015aa7a8d626 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:47.775182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 570/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:47 2022] GET /identity/v3/role_assignments?user.id=e281d5e5e56c4a3e9535cf961d99f731&scope.project.id=a6bc80a223a14a04b4dce9b3d206da28&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:49.660328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdf36b-400c-42b1-a8d5-74df37bf819c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:49.660877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdf36b-400c-42b1-a8d5-74df37bf819c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:49.660877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdf36b-400c-42b1-a8d5-74df37bf819c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:49.662227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 577/1147] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:49.667532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:49.667910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:49.668076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:49.685741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:49.686908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:49.752186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 571/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:50.020555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdf36b-400c-42b1-a8d5-74df37bf819c None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:50.064457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:50.121509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdf36b-400c-42b1-a8d5-74df37bf819c None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:50.154765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:50.155381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:50.155655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:50.155921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:50.166206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:50.171020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:50.172180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:50.173786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 578/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:50.178789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:50.257744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:50.279164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5cfaf-1108-46b4-b689-5b1b105f62be None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:50.313494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:50.314090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:50.314369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:50.314642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:50.315302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:50.315601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:50.316985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:50.324768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 572/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:50.330765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:50.428079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4b8f6-cbdd-44be-a022-040c5859a68f None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:50.429586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:50.465534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:50.466184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:50.466460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:50.466734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] PATH_INFO: `/v3/users/e281d5e5e56c4a3e9535cf961d99f731` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:50.467872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] RBAC: Authorizing `identity:get_user(user_id=e281d5e5e56c4a3e9535cf961d99f731)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:50.469876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:50.471899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 579/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:50 2022] GET /identity/v3/users/e281d5e5e56c4a3e9535cf961d99f731 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:50.477425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:50.575353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9ba8-dbba-4bff-bcd1-31042cb50243 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:50.608278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacaee5-f288-497d-b352-7ba33161f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:50.608913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacaee5-f288-497d-b352-7ba33161f111 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:50.609189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacaee5-f288-497d-b352-7ba33161f111 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:50.609458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacaee5-f288-497d-b352-7ba33161f111 None admin] PATH_INFO: `/v3/projects/a6bc80a223a14a04b4dce9b3d206da28` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:50.610597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacaee5-f288-497d-b352-7ba33161f111 None admin] RBAC: Authorizing `identity:get_project(project_id=a6bc80a223a14a04b4dce9b3d206da28)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:50.612001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacaee5-f288-497d-b352-7ba33161f111 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:50.613632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 573/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:50 2022] GET /identity/v3/projects/a6bc80a223a14a04b4dce9b3d206da28 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:50.619537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:50.713734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ae63-602c-493d-8b44-9dedefaae98d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:50.746182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fd36e-074f-4892-a79b-885d09db868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:50.746820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd36e-074f-4892-a79b-885d09db868b None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:50.747095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd36e-074f-4892-a79b-885d09db868b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:50.747366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd36e-074f-4892-a79b-885d09db868b None admin] PATH_INFO: `/v3/projects/a6bc80a223a14a04b4dce9b3d206da28/users/e281d5e5e56c4a3e9535cf961d99f731/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:50.749240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fd36e-074f-4892-a79b-885d09db868b None admin] RBAC: Authorizing `identity:create_grant(project_id=a6bc80a223a14a04b4dce9b3d206da28, user_id=e281d5e5e56c4a3e9535cf961d99f731, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:50.750666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fd36e-074f-4892-a79b-885d09db868b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:50.866294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 580/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:46:50 2022] PUT /identity/v3/projects/a6bc80a223a14a04b4dce9b3d206da28/users/e281d5e5e56c4a3e9535cf961d99f731/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:46:52.778492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e05509-3edf-4f77-a09b-c7a1bfe47f0c None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:52.779017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e05509-3edf-4f77-a09b-c7a1bfe47f0c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:52.779017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e05509-3edf-4f77-a09b-c7a1bfe47f0c None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:52.780392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 574/1154] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:52.785701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:52.786079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:52.786246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:52.804949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:52.806135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:52.829448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:52.870875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 581/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:53.142527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e05509-3edf-4f77-a09b-c7a1bfe47f0c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:53.154197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:53.243773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e05509-3edf-4f77-a09b-c7a1bfe47f0c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:53.277626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:53.278248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:53.278527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:53.278798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:53.289927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:46:53.294833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:53.295964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:46:53.297020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 575/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:53.302847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:53.399359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c1803-e56d-4cd1-a229-bde13e487158 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:53.432685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:53.433278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:53.433561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:53.433837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:53.434491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:53.434797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:53.436144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:53.443897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 582/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:53.449712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:53.516730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:53.549993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca697a-abc5-4b00-b7b8-1518db2c9ba0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:53.582624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:53.583232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:53.583510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:53.583779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] PATH_INFO: `/v3/users/e281d5e5e56c4a3e9535cf961d99f731` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:53.584905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] RBAC: Authorizing `identity:get_user(user_id=e281d5e5e56c4a3e9535cf961d99f731)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:53.586323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:53.587886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 576/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:46:53 2022] GET /identity/v3/users/e281d5e5e56c4a3e9535cf961d99f731 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:53.593712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:53.618279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:53.690923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a6a09-ea72-4a0c-8b34-e24b3cf21ba2 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:53.723828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:53.724445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:53.724726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:53.725000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:53.790754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:53.792189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:53.815897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:53.889541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 583/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:46:53.895771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:53.993673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060c0a-adb6-4adb-9be4-d516310cfb90 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:54.025529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b29b92-711e-4999-a708-c7c3bbefef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:54.026135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b29b92-711e-4999-a708-c7c3bbefef32 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:54.026410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b29b92-711e-4999-a708-c7c3bbefef32 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:54.026678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b29b92-711e-4999-a708-c7c3bbefef32 None admin] PATH_INFO: `/v3/projects/a6bc80a223a14a04b4dce9b3d206da28` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:54.027773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b29b92-711e-4999-a708-c7c3bbefef32 None admin] RBAC: Authorizing `identity:get_project(project_id=a6bc80a223a14a04b4dce9b3d206da28)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:54.029214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b29b92-711e-4999-a708-c7c3bbefef32 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:54.030811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 577/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:46:53 2022] GET /identity/v3/projects/a6bc80a223a14a04b4dce9b3d206da28 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:54.037374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:54.132724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefeb2f-f5fd-4399-a778-ddd4d532f926 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:54.153940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:54.165413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:54.166004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:54.166282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:54.166555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:54.167272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=a6bc80a223a14a04b4dce9b3d206da28, user.id=e281d5e5e56c4a3e9535cf961d99f731) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:54.167577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:54.169003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4a2ca-bc3b-4d14-95c4-8aa43d4b6bf3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:54.175048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 584/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:46:54 2022] GET /identity/v3/role_assignments?user.id=e281d5e5e56c4a3e9535cf961d99f731&scope.project.id=a6bc80a223a14a04b4dce9b3d206da28&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:56.076991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3fd8-9dfe-4793-897f-90128056ca70 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:56.077517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3fd8-9dfe-4793-897f-90128056ca70 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:56.077517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3fd8-9dfe-4793-897f-90128056ca70 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:56.078889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 578/1162] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:56.084373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:56.084795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:56.084922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:56.102769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:56.103926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:56.168788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 585/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:56.437369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f3fd8-9dfe-4793-897f-90128056ca70 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:56.441475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:56.537740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f3fd8-9dfe-4793-897f-90128056ca70 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:56.569373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:56.569965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:56.570240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:56.570509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:56.635401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:56.636868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:56.639372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:56.736734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 579/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:46:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:46:56.742597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:56.838695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5654a-a109-4768-9572-b554f3300f1a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:56.854886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:56.871698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:56.874235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:56.874235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:56.874235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:56.879490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:56.881000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-a5d1113d-3fb3-4515-8be1-adfafa9ab1fe None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:46:56.893309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:46:56.894798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:46:56.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 586/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:46:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:46:56.900659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:56.997557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415307af-f652-4c87-ad0a-1bb4f2a7343c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:56.999747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:46:57.031656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:57.032246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:57.032548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:57.032823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:57.033459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:46:57.033758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:57.035213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adcdf8-f776-4efd-b0b2-249cf3ef4583 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:57.042768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 580/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 1 17:46:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:46:58.920454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59f58b-b2a4-411a-8dd4-844284aef78b None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:58.920984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59f58b-b2a4-411a-8dd4-844284aef78b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:58.921095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59f58b-b2a4-411a-8dd4-844284aef78b None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:58.923022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 587/1167] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:46:58.928138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c024c-87a3-4c19-ba53-32614c27d82a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:58.928534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c024c-87a3-4c19-ba53-32614c27d82a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:58.928730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c024c-87a3-4c19-ba53-32614c27d82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:58.947323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-959c024c-87a3-4c19-ba53-32614c27d82a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:46:58.948542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-959c024c-87a3-4c19-ba53-32614c27d82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:46:59.016150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 581/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:46:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:46:59.286563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59f58b-b2a4-411a-8dd4-844284aef78b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:46:59.386817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59f58b-b2a4-411a-8dd4-844284aef78b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:46:59.419618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ab655-0451-4f9b-b00d-3546fc9394e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:46:59.420219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ab655-0451-4f9b-b00d-3546fc9394e6 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:46:59.420537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ab655-0451-4f9b-b00d-3546fc9394e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:46:59.420814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ab655-0451-4f9b-b00d-3546fc9394e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:46:59.421484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ab655-0451-4f9b-b00d-3546fc9394e6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:46:59.423409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ab655-0451-4f9b-b00d-3546fc9394e6 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:46:59.529336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 588/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Thu Sep 1 17:46:59 2022] POST /identity/v3/domains => generated 267 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:01.436211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1a4d-1443-4671-bc17-2b948b0aad0f None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:01.436796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1a4d-1443-4671-bc17-2b948b0aad0f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:01.436796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1a4d-1443-4671-bc17-2b948b0aad0f None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:01.438426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 582/1170] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:47:01.443721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:01.444122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:01.444268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:01.461968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:01.463135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:01.491504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:01.527340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 589/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:47:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:47:01.798023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1a4d-1443-4671-bc17-2b948b0aad0f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:01.897591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:01.902197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1a4d-1443-4671-bc17-2b948b0aad0f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:01.935887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867f605-6f06-480d-a396-6f06b51b3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:01.936542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f605-6f06-480d-a396-6f06b51b3176 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:01.936827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f605-6f06-480d-a396-6f06b51b3176 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:01.937101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f605-6f06-480d-a396-6f06b51b3176 None admin] PATH_INFO: `/v3/domains/39b61896184e4e088cfba6250efb1ac7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:01.947189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f605-6f06-480d-a396-6f06b51b3176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=39b61896184e4e088cfba6250efb1ac7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:01.948648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f605-6f06-480d-a396-6f06b51b3176 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:01.950234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 583/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Thu Sep 1 17:47:01 2022] GET /identity/v3/domains/39b61896184e4e088cfba6250efb1ac7 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:01.956799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:02.051254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e5159-00d4-429c-9173-46ccea5c6b70 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:02.084031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575dfda-86d0-41fa-a6a3-b7cf4eeb1ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:02.084665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575dfda-86d0-41fa-a6a3-b7cf4eeb1ae8 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:02.084946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575dfda-86d0-41fa-a6a3-b7cf4eeb1ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:02.085218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575dfda-86d0-41fa-a6a3-b7cf4eeb1ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:02.085858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575dfda-86d0-41fa-a6a3-b7cf4eeb1ae8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:02.087264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575dfda-86d0-41fa-a6a3-b7cf4eeb1ae8 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:02.175083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 590/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:47:01 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:04.210276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ae97-cbd7-41c8-aec5-67c7e322f4ce None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:04.210805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ae97-cbd7-41c8-aec5-67c7e322f4ce None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:04.210805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ae97-cbd7-41c8-aec5-67c7e322f4ce None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:04.212147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 584/1174] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:47:04.217461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:04.217879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:04.218016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:04.236634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:04.237972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:04.250981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:04.303578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 591/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:47:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:47:04.572058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ae97-cbd7-41c8-aec5-67c7e322f4ce None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:04.670913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ae97-cbd7-41c8-aec5-67c7e322f4ce None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:04.703996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f9aa2-6a0a-484d-8ae4-8d368a519ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:04.704678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f9aa2-6a0a-484d-8ae4-8d368a519ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:04.704959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f9aa2-6a0a-484d-8ae4-8d368a519ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:04.705231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f9aa2-6a0a-484d-8ae4-8d368a519ba0 None admin] PATH_INFO: `/v3/domains/39b61896184e4e088cfba6250efb1ac7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:04.706345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f9aa2-6a0a-484d-8ae4-8d368a519ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=39b61896184e4e088cfba6250efb1ac7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:04.707765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f9aa2-6a0a-484d-8ae4-8d368a519ba0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:04.709370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 585/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Thu Sep 1 17:47:04 2022] GET /identity/v3/domains/39b61896184e4e088cfba6250efb1ac7 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:04.715127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:04.811746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0b617-0b95-4620-85a6-067d3c0ae634 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:04.844417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4325f13-8d39-4d30-8165-494ca762387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:04.844981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4325f13-8d39-4d30-8165-494ca762387a None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:04.845261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4325f13-8d39-4d30-8165-494ca762387a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:04.845534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4325f13-8d39-4d30-8165-494ca762387a None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:04.846183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4325f13-8d39-4d30-8165-494ca762387a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:04.847581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4325f13-8d39-4d30-8165-494ca762387a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:04.954601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 592/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:47:04 2022] POST /identity/v3/users => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:47:06.885303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8dc05-ca0d-49be-b942-54a67b7e90fc None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:06.885831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8dc05-ca0d-49be-b942-54a67b7e90fc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:06.885831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8dc05-ca0d-49be-b942-54a67b7e90fc None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:06.887205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 586/1178] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:47:06.892469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:06.892900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:06.893048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:06.900454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:06.911815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:06.913016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:06.978119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 593/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:47:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:47:07.248685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8dc05-ca0d-49be-b942-54a67b7e90fc None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:07.277157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:07.350909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8dc05-ca0d-49be-b942-54a67b7e90fc None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:07.383295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:07.383914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:07.384193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:07.384505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:07.395678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:47:07.400561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:47:07.405989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:47:07.408643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 587/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:47:07.409563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:07.514303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e741d7b-f77d-495f-b3c5-03e3839c81f1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:07.547152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:07.547820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:07.548095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:07.548395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:07.549058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:07.549359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:07.550704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:07.552005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:07.558894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 594/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:07.565304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:07.649395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:07.663749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95fdfb-63cc-44fa-a7d7-92932c83d070 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:07.696870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:07.697484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:07.697765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:07.698039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] PATH_INFO: `/v3/users/ff60386fe00c4127850b53dd6429190b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:07.711307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] RBAC: Authorizing `identity:get_user(user_id=ff60386fe00c4127850b53dd6429190b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:07.712783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:07.714376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 588/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:47:07 2022] GET /identity/v3/users/ff60386fe00c4127850b53dd6429190b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:07.720849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:07.850097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2f45-a9ea-4e5e-934a-bc0511a51deb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:07.890985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:07.891649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:07.891927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:07.892201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:07.951306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:07.957708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:07.959143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:08.054358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 595/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:47:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:47:08.060546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:08.168326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b61dc-cf66-4385-b3d9-2c054613643c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:08.202202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f158f0c-2a84-416e-b647-580b9b36e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:08.202810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f158f0c-2a84-416e-b647-580b9b36e10e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:08.203086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f158f0c-2a84-416e-b647-580b9b36e10e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:08.203429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f158f0c-2a84-416e-b647-580b9b36e10e None admin] PATH_INFO: `/v3/projects/9a16ae19fd674c52ae15f1c907500f1e` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:08.204661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f158f0c-2a84-416e-b647-580b9b36e10e None admin] RBAC: Authorizing `identity:get_project(project_id=9a16ae19fd674c52ae15f1c907500f1e)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:08.206066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f158f0c-2a84-416e-b647-580b9b36e10e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:08.207678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 589/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:47:08 2022] GET /identity/v3/projects/9a16ae19fd674c52ae15f1c907500f1e => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:08.214099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:08.309463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:08.315085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962a82d-a15e-4e4c-a4f3-1f9e553c8edd None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:08.348871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:08.349469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:08.349747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:08.350019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:08.350736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=9a16ae19fd674c52ae15f1c907500f1e, user.id=ff60386fe00c4127850b53dd6429190b) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:08.351041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:08.352460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9614-3644-4cd5-9163-ad0a6bb67938 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:08.357627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 596/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:47:08 2022] GET /identity/v3/role_assignments?user.id=ff60386fe00c4127850b53dd6429190b&scope.project.id=9a16ae19fd674c52ae15f1c907500f1e&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:10.242768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf0f5-d33c-4f2c-b0e0-235e19b59282 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.243310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf0f5-d33c-4f2c-b0e0-235e19b59282 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.243310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf0f5-d33c-4f2c-b0e0-235e19b59282 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.244693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 590/1186] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:47:10.249764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.250156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.250576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.269303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:10.270488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:10.336527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 597/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:47:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:47:10.605012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bf0f5-d33c-4f2c-b0e0-235e19b59282 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.619603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:10.705884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bf0f5-d33c-4f2c-b0e0-235e19b59282 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:10.738657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:10.739265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.739544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.739816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.749576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:47:10.754903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:47:10.756004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:47:10.757220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 591/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:47:10.761934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.847561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:10.857944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce622fff-aa38-4527-a10c-d27cd2cb2c82 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:10.890197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:10.891168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:10.891385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:10.891657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:10.892339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:10.892646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:10.894236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:10.901972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 598/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:47:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:10.907658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:10.976562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.005385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bb2e2-6be6-47c8-aa99-8b3e9b55d7c0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.038500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.039105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.039383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.039653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] PATH_INFO: `/v3/users/ff60386fe00c4127850b53dd6429190b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.040787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] RBAC: Authorizing `identity:get_user(user_id=ff60386fe00c4127850b53dd6429190b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.042202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.043753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 592/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:47:10 2022] GET /identity/v3/users/ff60386fe00c4127850b53dd6429190b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:11.050207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.148632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2399-9e28-4c07-8f71-b2ca640be465 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.181316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2057dcf-ba8d-4e26-b076-bc00418d0d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.181931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2057dcf-ba8d-4e26-b076-bc00418d0d4f None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.182208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2057dcf-ba8d-4e26-b076-bc00418d0d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.182479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2057dcf-ba8d-4e26-b076-bc00418d0d4f None admin] PATH_INFO: `/v3/projects/9a16ae19fd674c52ae15f1c907500f1e` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.183568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2057dcf-ba8d-4e26-b076-bc00418d0d4f None admin] RBAC: Authorizing `identity:get_project(project_id=9a16ae19fd674c52ae15f1c907500f1e)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.184999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2057dcf-ba8d-4e26-b076-bc00418d0d4f None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.186598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 599/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:47:11 2022] GET /identity/v3/projects/9a16ae19fd674c52ae15f1c907500f1e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:11.192655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:11.290754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a0d7-544e-4e7e-8207-87aaf33d0f95 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:11.323520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d395c1-cd62-4046-af10-d7e3886fe1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:11.324157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d395c1-cd62-4046-af10-d7e3886fe1e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:11.324471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d395c1-cd62-4046-af10-d7e3886fe1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:11.324742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d395c1-cd62-4046-af10-d7e3886fe1e9 None admin] PATH_INFO: `/v3/projects/9a16ae19fd674c52ae15f1c907500f1e/users/ff60386fe00c4127850b53dd6429190b/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:11.326591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d395c1-cd62-4046-af10-d7e3886fe1e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a16ae19fd674c52ae15f1c907500f1e, user_id=ff60386fe00c4127850b53dd6429190b, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:11.327990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d395c1-cd62-4046-af10-d7e3886fe1e9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:11.435904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:11.440322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 593/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:47:11 2022] PUT /identity/v3/projects/9a16ae19fd674c52ae15f1c907500f1e/users/ff60386fe00c4127850b53dd6429190b/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:47:13.345129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6eec0-5148-4dfb-b7b9-ef9a36c2f72e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:13.345645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6eec0-5148-4dfb-b7b9-ef9a36c2f72e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:13.345737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6eec0-5148-4dfb-b7b9-ef9a36c2f72e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:13.347019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 600/1193] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:47:13.352009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:13.352463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:13.352585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:13.370409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:47:13.371575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:47:13.436297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 594/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:47:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:47:13.709610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6eec0-5148-4dfb-b7b9-ef9a36c2f72e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:13.731412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:13.813462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6eec0-5148-4dfb-b7b9-ef9a36c2f72e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:13.846048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:13.846662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:13.846937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:13.847206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:13.856921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:47:13.862377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:47:13.863600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:47:13.864584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 601/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:47:13.869680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:13.978218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851c7ba-2462-423f-8eef-cf17c163d008 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:14.007026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:14.012591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:14.013197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:14.013478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:14.013752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:14.014421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:14.014722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:14.016123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:14.024366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 595/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:47:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:14.030347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:14.108263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:14.127256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e6e8-440a-49f0-8476-546f8aceedff None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:14.161220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:14.161839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:14.162115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:14.162385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] PATH_INFO: `/v3/users/ff60386fe00c4127850b53dd6429190b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:14.163489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] RBAC: Authorizing `identity:get_user(user_id=ff60386fe00c4127850b53dd6429190b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:14.164933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:14.166538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 602/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 1 17:47:14 2022] GET /identity/v3/users/ff60386fe00c4127850b53dd6429190b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:14.173036 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:14.272015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa39a4-56a8-4017-8f1a-0b50db827f37 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:14.306277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:14.306878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:14.307157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:14.307449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:14.364537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:14.373583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:14.375045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:14.471402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 596/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:47:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:47:14.477568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:14.580561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eeea6-e766-467e-8b80-66a027fb0f91 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:14.612963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787f2c8-ecdf-4aaa-b0a2-c214f5f1277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:14.613580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787f2c8-ecdf-4aaa-b0a2-c214f5f1277a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:14.613855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787f2c8-ecdf-4aaa-b0a2-c214f5f1277a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:14.614123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787f2c8-ecdf-4aaa-b0a2-c214f5f1277a None admin] PATH_INFO: `/v3/projects/9a16ae19fd674c52ae15f1c907500f1e` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:14.615211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787f2c8-ecdf-4aaa-b0a2-c214f5f1277a None admin] RBAC: Authorizing `identity:get_project(project_id=9a16ae19fd674c52ae15f1c907500f1e)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:14.616646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787f2c8-ecdf-4aaa-b0a2-c214f5f1277a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:14.618253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 603/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 1 17:47:14 2022] GET /identity/v3/projects/9a16ae19fd674c52ae15f1c907500f1e => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:14.624386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:47:14.712979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:14.721826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516016a-d051-4770-9f06-ca6294ba543a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:47:14.754511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:47:14.755101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:47:14.755401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:47:14.755673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:47:14.756427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=9a16ae19fd674c52ae15f1c907500f1e, user.id=ff60386fe00c4127850b53dd6429190b) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:47:14.756736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:47:14.758148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50874a-f1d0-4d7b-8903-4d301538f9b0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:47:14.763787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 597/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:47:14 2022] GET /identity/v3/role_assignments?user.id=ff60386fe00c4127850b53dd6429190b&scope.project.id=9a16ae19fd674c52ae15f1c907500f1e&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:47:24.608978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:47:24.761252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=22 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:31.047428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419cd8f-eae6-4d2f-ac46-8bdc18c73ac8 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:31.048027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419cd8f-eae6-4d2f-ac46-8bdc18c73ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:31.048027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419cd8f-eae6-4d2f-ac46-8bdc18c73ac8 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:31.049333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 604/1201] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:31.054699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:31.059316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:31.059316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:31.073872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:31.075048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:31.140861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 598/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:31.414693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419cd8f-eae6-4d2f-ac46-8bdc18c73ac8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:31.517460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419cd8f-eae6-4d2f-ac46-8bdc18c73ac8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:31.551576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:31.552194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:31.552506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:31.552780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:31.563533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:31.565000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:31.579456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:31.581025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:31.582045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:31.582045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:31.582045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:31.582045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 605/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:31.586696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:31.685418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a972fa3-8d77-4a03-9a74-52d7237e586d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:31.712750 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:31.718412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:31.719012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:31.719294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:31.719609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:31.720253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:31.723378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:31.723378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850a264-9cb1-4ad5-8bd0-73c83c365276 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:31.733470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 599/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:31.739467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:31.791576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:31.836421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf4d9-10a9-44d0-a1f5-889d2b57661c None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:31.869815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5c786-ea11-40d6-b344-553540147bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:31.870411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c786-ea11-40d6-b344-553540147bee None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:31.870689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c786-ea11-40d6-b344-553540147bee None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:31.870960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c786-ea11-40d6-b344-553540147bee None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:31.871606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5c786-ea11-40d6-b344-553540147bee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:31.873123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5c786-ea11-40d6-b344-553540147bee None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:31.981031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 606/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 1 17:48:31 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:48:33.910018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7a58e-56ab-4317-ac1e-3d36478188b2 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:33.910547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7a58e-56ab-4317-ac1e-3d36478188b2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:33.910547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7a58e-56ab-4317-ac1e-3d36478188b2 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:33.911907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 600/1206] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:33.917040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:33.920819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:33.920819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:33.935508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:33.936709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:34.010140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 607/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:34.277290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7a58e-56ab-4317-ac1e-3d36478188b2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:34.385467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7a58e-56ab-4317-ac1e-3d36478188b2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:34.419131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:34.419753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:34.420036 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:34.420355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:34.431282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:48:34.436279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:34.437312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:34.438379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 601/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:48:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:34.443648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:34.483901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:34.543687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c99e-f58a-4472-872d-571eb3e69019 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:34.576039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:34.576665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:34.576944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:34.577213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:34.577866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:34.578200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:34.579505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:34.587609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 608/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:48:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:34.593369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:34.644376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:34.694150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6b84-aa77-4134-9a86-6a38efe262ea None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:34.725621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:34.726237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:34.726512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:34.726781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:34.738152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:34.739591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:34.754109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:34.755086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:34.756086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:34.756086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:34.756086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:34.756086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 602/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:34.761188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:34.856031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:34.858791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9260a00-feb4-4378-8cfd-67916b3d9b17 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:34.890984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:34.891572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:34.891849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:34.892120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:34.892780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:34.893077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:34.894404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:34.906203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 609/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:34.917700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:35.011006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bd225-c99b-4277-9b01-923ae5cb2fcb None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:35.012624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:35.045903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:35.049604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:35.049604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:35.049604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:35.054377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:35.055993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:48:35.066866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:35.068003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:35.069269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 603/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 17:48:35.073908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:35.173582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebf4c5-67e6-4f61-8922-d4678fc2ea82 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:35.207504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:35.209353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:35.209353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:35.209353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:35.209671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:35.209765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:35.211152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:35.228936 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 610/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 1 17:48:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:35.234711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:35.335009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bba0bd-1cd0-459c-8866-d6eb05b28657 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:35.367818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:35.368452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:35.368733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:35.369003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:35.398585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:35.434818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:35.436194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:35.535983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 604/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:48:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:48:35.541856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:35.560656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:35.639768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec45de-1f36-4c14-87a4-acb56b77d088 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:35.674899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:35.675517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:35.675793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:35.676062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:35.686445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:35.687802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:35.703070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:35.704242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:35.705225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:35.705225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:35.705225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:35.705225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 611/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:35.709907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:35.769978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:35.808058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f003129-8770-4482-becb-d994003d2da5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:35.842982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:35.843580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:35.843855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:35.844124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:35.844905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:35.845133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:35.846471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:35.857718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 605/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:35.863393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:35.931692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:35.961900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf84235-ebb5-49d4-ada1-99fd62cd2067 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:35.994989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:35.995607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:35.995887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:35.996160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:36.002807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:36.004219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:48:36.015943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:36.022464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:36.023623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 612/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:36.024269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:36.120940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40184d29-419f-4047-ac3d-56de34ec8f26 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:36.142303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:36.154195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:36.159809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:36.159809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:36.159809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:36.159809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:36.159809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:36.159809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d648564-3841-4937-b9f7-ad03e372deba None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:36.170348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 606/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:48:36.176565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:36.275156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed5b39-f2b8-4438-9ec9-b04ab0deabca None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:36.303927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:36.310493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:36.311198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:36.311443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:36.311714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:36.312463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=4c88695a14384a0b85bb20a0589692bb) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:36.312766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:36.314237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b8b12-b2dd-4524-9d8f-d7c9d1ec8277 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:36.319673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 613/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:48:36 2022] GET /identity/v3/role_assignments?user.id=4c88695a14384a0b85bb20a0589692bb&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:38.203144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb65e26-9653-46b6-84be-e6a8af7052a1 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:38.203675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb65e26-9653-46b6-84be-e6a8af7052a1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:38.203675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb65e26-9653-46b6-84be-e6a8af7052a1 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:38.205063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 607/1220] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:38.210453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:38.210873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:38.210993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:38.229939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:38.231102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:38.296599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 614/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:38.575204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb65e26-9653-46b6-84be-e6a8af7052a1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:38.677991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb65e26-9653-46b6-84be-e6a8af7052a1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:38.712930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:38.713552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:38.713831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:38.714102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:38.724238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:48:38.729314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:38.734484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:38.735596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 608/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:38.737025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:38.834751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddeb11-6044-4ed5-bda9-1b447d51076b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:38.857454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:38.868692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:38.869297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:38.869658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:38.869937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:38.870599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:38.870899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:38.872239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:38.880066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 615/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:48:38.886486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:38.930049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:38.990502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f091-1b1b-4281-98d6-16c238925e95 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:39.036320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:39.037261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:39.037677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:39.038083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:39.051866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:39.054062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:39.071251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:39.072446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:39.073759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:39.073759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:39.073759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:39.073759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 609/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:39.078224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:39.178476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445cd6-41a7-46db-b81a-3fb14fea27fa None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:39.211526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:39.212507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:39.212793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:39.213070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:39.213714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:39.214018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:39.215368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:39.226730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 616/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:39.233157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:39.329224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:39.335799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82dd8d-f3d2-49b3-bfd2-b41ab33d3cdd None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:39.374873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:39.375791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:39.376210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:39.376661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:39.385379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:39.386824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:48:39.398372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:39.400437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:39.401680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 610/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:39.406142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:39.453127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:39.504276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a9a02-d403-4570-882a-3a6aa026d1a7 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:39.537569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:39.538163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:39.538586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:39.538856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:39.539504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:39.540560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:39.542108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:39.559083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 617/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 1 17:48:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:39.564948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:39.663906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853d5d7-75e2-4fab-838e-e3fdb7f8a58d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:39.698959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:39.699588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:39.699869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:39.700142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:39.715659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:39.715659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:39.726744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:39.727912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:39.729238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:39.729238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:39.729238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:39.729238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 611/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:39.734386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:39.834681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99f5e-b0a0-4a2e-a286-871441b1381d None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:39.837701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:39.867350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:39.867994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:39.868273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:39.868580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:39.869224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:39.869525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:39.870861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:39.881671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 618/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:39.888002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:39.898065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:39.989691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416da0c9-7af4-44cc-aac0-5d7ff0f0087d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:40.023446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:40.024070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:40.025010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:40.025526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:40.032059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:40.033610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:48:40.045827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:40.047003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:40.048144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:40.051722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 612/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:48:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:40.052900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:40.152595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc5045-c4a2-4a6d-8746-442345cba2b3 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:40.187127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:40.187725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:40.188001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:40.188268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:40.188985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:40.189284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:40.190751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8dcfb-8d0f-4337-9944-95df37234330 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:40.203174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 619/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:40.209305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:40.276835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:40.311497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b874f0-adf9-4f9a-a442-e93879d2620c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:40.345682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad08dc-f5be-4034-8d56-b17649e26a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:40.346330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad08dc-f5be-4034-8d56-b17649e26a8d None admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:40.346611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad08dc-f5be-4034-8d56-b17649e26a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:40.346886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad08dc-f5be-4034-8d56-b17649e26a8d None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/4c88695a14384a0b85bb20a0589692bb/roles/05ffd5a9519a4b53b71f4a47095d060c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:40.361203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad08dc-f5be-4034-8d56-b17649e26a8d None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=4c88695a14384a0b85bb20a0589692bb, role_id=05ffd5a9519a4b53b71f4a47095d060c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:40.362672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad08dc-f5be-4034-8d56-b17649e26a8d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:40.424902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 613/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:48:40 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/4c88695a14384a0b85bb20a0589692bb/roles/05ffd5a9519a4b53b71f4a47095d060c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:48:42.364961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daffe7-2a28-451b-be5f-9d8aac5525e5 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:42.365493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daffe7-2a28-451b-be5f-9d8aac5525e5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:42.365493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daffe7-2a28-451b-be5f-9d8aac5525e5 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:42.366831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 620/1233] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:48:42.372518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:42.372927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:42.373069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:42.391611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:42.392830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:42.459075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 614/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:42.728500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daffe7-2a28-451b-be5f-9d8aac5525e5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:42.753368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:42.828295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daffe7-2a28-451b-be5f-9d8aac5525e5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:42.861440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:42.862055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:42.862330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:42.862598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:42.873012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:48:42.877843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:42.884374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:42.885445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 621/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 1 17:48:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:42.886234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:42.931818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:42.983899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19477f-3a5b-4470-a76a-f780bf9b4eeb None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:43.018804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:43.019406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:43.019684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:43.019952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:43.020654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] RBAC: Adding query filter params (name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:43.021220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:43.022575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:43.030542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 615/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 1 17:48:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:43.036407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:43.119138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:43.134443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60eae7-a901-41be-88ca-29e8567a4501 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:43.169435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:43.170090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:43.170370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:43.170650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:43.180606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:43.182126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:43.196464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:43.197587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:43.198641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:43.198641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:43.198641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:43.198641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 622/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:43.203790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:43.305499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957d86c-1473-47b7-91e3-f8102a8ee409 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:43.314176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:43.338637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:43.339244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:43.339524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:43.339800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:43.340495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:43.340798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:43.342203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:43.354315 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 616/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:43.360207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:43.465386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b059e3-0730-4040-9901-9abbf48680a4 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:43.499822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:43.500476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:43.500754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:43.501023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:43.503758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:43.508646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:43.510050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:48:43.521608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:43.522790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:43.524006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 623/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:43.529490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:43.631161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be3ea3-4fd9-48c9-8b5b-4fc1787238d0 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:43.666326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:43.666936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:43.667213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:43.667485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:43.668215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:43.668554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:43.669978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:43.687471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 617/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 1 17:48:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:43.693348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:43.806841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b53c86-ada6-42c5-92a0-e8553858d486 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:43.844019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:43.844653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:43.844932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:43.845202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:43.905739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:43.912349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:43.913787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:44.011375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 624/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:48:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:48:44.017305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:44.045319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:44.123886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4e67-6979-4563-9f3c-fd32abf4e0b9 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:44.161821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:44.162445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:44.162724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:44.162993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:44.172800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:44.174229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:44.189993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:44.191176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:44.192214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:44.192214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:44.192214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:44.192214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 618/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:44.197473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:44.291583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:44.297152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0717fd-65ec-4c73-8083-f8555d138fef None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:44.331034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:44.331634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:44.331913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:44.332186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:44.332854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:44.333152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:44.334492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:44.345551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 625/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 17:48:44.351416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:44.431356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:44.453280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2d942-b9ca-4149-b177-b7ca89d1f0a8 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:44.488461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:44.489087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:44.489364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:44.489633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:44.497131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:44.498552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:48:44.510789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:44.511951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:44.519634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 619/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:48:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:44.520277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:44.617581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fe88e-e057-4048-b63d-af7c145a04bf None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:44.651432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:44.652030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:44.652333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:44.652607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:44.653293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:44.653598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:44.654993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838e1f5-0da1-486f-8306-ed984d15ddca None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:44.666757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 626/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:48:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:44.672908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:44.774093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3f15e-b522-441d-8725-29e9195b9a1a None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:44.807165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:44.807763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:44.808038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:44.808304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:44.809047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] RBAC: Adding query filter params (role.id=05ffd5a9519a4b53b71f4a47095d060c, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=4c88695a14384a0b85bb20a0589692bb) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:44.813591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:44.813591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ccdaf-ace1-4d98-8cd9-94f1c8bfe10c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:44.816577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 620/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:48:44 2022] GET /identity/v3/role_assignments?user.id=4c88695a14384a0b85bb20a0589692bb&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=05ffd5a9519a4b53b71f4a47095d060c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:46.781525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb27a4-dfd0-49ea-b4a6-e08303d81d8f None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:46.782105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb27a4-dfd0-49ea-b4a6-e08303d81d8f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:46.782105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb27a4-dfd0-49ea-b4a6-e08303d81d8f None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:46.783410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 627/1247] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:46.788547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:46.788987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:46.789090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:46.796094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:46.808698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:46.811670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:46.876680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 621/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:47.147600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb27a4-dfd0-49ea-b4a6-e08303d81d8f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:47.166904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:47.248232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb27a4-dfd0-49ea-b4a6-e08303d81d8f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:47.284373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:47.285850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:47.285850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:47.285850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:47.295611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:48:47.300570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:47.301790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:47.302943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 628/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:48:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:47.308671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:47.406311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c8ef-be13-4e2a-9d92-f77d44f4afe5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:47.435471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:47.440431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:47.441045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:47.441322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:47.441594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:47.442260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:47.442561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:47.443921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:47.452862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 622/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:47.458726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:47.535083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:47.557775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666e81d-f9cd-4a6e-9732-6bb039226a33 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:47.590093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:47.590709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:47.590985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:47.591255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:47.601372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:47.602919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:47.616774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:47.621091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:47.622223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:47.622223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:47.622223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:47.622223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 629/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:47.625722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:47.726229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da98c6-b822-4597-a4e9-b6438c433d05 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:47.765672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:47.766556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:47.766980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:47.767389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:47.768397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:47.772640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:47.772640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:47.789759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 623/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:47.796129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:47.895175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfeff6-f0cd-4508-97d0-f891178f6302 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:47.922145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:47.929632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:47.930890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:47.931186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:47.931460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:47.938712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:47.940121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:48:47.951140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:47.952305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:47.953618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 630/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:47.958391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:48.014370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:48.059041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f34256-cca6-4619-9ba2-03f5e62657e5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:48.092965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:48.093570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:48.093854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:48.094135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:48.094862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:48.095169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:48.096609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:48.115201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 624/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 1 17:48:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:48.124784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:48.237422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25159f33-aa49-4be6-a15e-12b8884446cb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:48.270471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:48.271271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:48.271545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:48.271811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:48.326134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:48.338656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:48.340093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:48.437898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 631/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:48:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:48:48.443827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:48.555589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cfda2-7296-4a37-82d4-a9a2dd4f1285 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:48.568616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:48.590657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:48.591584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:48.591700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:48.591934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:48.601658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:48.603080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:48.617881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:48.621216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:48.622359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:48.622359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:48.622359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:48.622359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 625/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:48.625343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:48.712866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:48.725127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef94369-ac4e-4767-a4d6-0327c97c6d1b None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:48.758660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:48.759264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:48.759540 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:48.760424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:48.761060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:48.761354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:48.762681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:48.775975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 632/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:48.781851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:48.881904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d1867-1353-4f1c-95a1-9a88817fedd3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:48.915223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:48.915841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:48.916209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:48.916515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:48.923017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:48.924479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:48:48.936408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:48.937506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:48.938630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 626/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:48.943460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:49.043457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46695-c731-4519-8de8-de23da56f809 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:49.077696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:49.078297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:49.078577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:49.078843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:49.079551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:49.079850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:49.081319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c460f-6b35-4137-915d-cde0f032f0b3 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:49.093224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 633/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:49.099674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:49.126079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:49.202973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c9d1-718b-4bdd-a0d1-8b474a8d0f65 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:49.236046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:49.236767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:49.237059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:49.237336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:49.238079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=4c88695a14384a0b85bb20a0589692bb) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:49.238392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:49.239855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e0a45-baa4-42a8-88b5-344582e0edb0 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:49.246378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 627/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:48:49 2022] GET /identity/v3/role_assignments?user.id=4c88695a14384a0b85bb20a0589692bb&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:51.135775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d7042-a695-4ffe-80d3-46970860400b None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.136299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d7042-a695-4ffe-80d3-46970860400b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.136299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d7042-a695-4ffe-80d3-46970860400b None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.137705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 634/1261] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:51.143033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.143447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.143558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.163375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:51.164618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:51.232004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 628/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:51.504288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d7042-a695-4ffe-80d3-46970860400b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:51.515875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:51.605085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d7042-a695-4ffe-80d3-46970860400b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:51.639524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:51.640151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.640463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.640737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.651710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:48:51.656667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:51.657787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:51.658724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 635/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:51.663785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:51.681828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:51.760347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78578b-6e19-4a84-9c3e-a8a98027bb8e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:51.794157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:51.797650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.797650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.797650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.797650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:51.797650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:51.797650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:51.805375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 629/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:51.811646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:51.876982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:51.909253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd6e0c-698d-4d8b-8693-0993ae1a11e9 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:51.941963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:51.942636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:51.942909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:51.943177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:51.953904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:51.955310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:51.968910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:51.970050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:51.971345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:51.971345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:51.971345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:51.971345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 636/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:51.975855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:52.056239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:52.077138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c308f-a468-4090-af87-42af2a06c008 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:52.111501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:52.112107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:52.112417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:52.112698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:52.113343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:52.113647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:52.115002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:52.127269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 630/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:52.133234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:52.236881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f638d8-4ff5-48fe-be97-33ed88962d3e None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:52.257419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:52.271837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:52.272541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:52.272977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:52.273248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:52.280486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:52.281895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:48:52.293606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:52.294758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:52.295827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 637/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:52.300580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:52.400906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893c75-3290-4220-901f-e0478460f437 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:52.435063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:52.435661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:52.435939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:52.436208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:52.436968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:52.437270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:52.438680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:52.447415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:52.455362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 631/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 1 17:48:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:52.461165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:52.567975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8131-d04d-4927-92da-1272f03248eb None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:52.600908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:52.601571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:52.601849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:52.602122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:52.613018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:52.619353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:52.628595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:52.629654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:52.630671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:52.630671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:52.630671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:52.630671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 638/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:52.635692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:52.733789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d458ac-ff51-4396-9d95-3b2ae13ff679 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:52.766454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:52.767050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:52.767330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:52.767603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:52.768241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:52.768568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:52.769927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:52.781737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 632/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:52.787448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:52.819026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:52.884604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14ea9-b8e3-4078-a156-0e39a76066ca None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:52.918557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:52.919180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:52.919456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:52.919727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:52.926667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:52.928154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:48:52.939137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:52.940168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:52.941379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 639/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:48:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:52.946144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:53.006046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:53.047860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5afac-abe4-4e79-ad21-88e87ad2a8e3 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:53.081791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:53.082399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:53.082682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:53.082950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:53.083647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:53.083950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:53.085396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c797d-89ef-48fe-bccd-5b3662bd2c91 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:53.098363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 633/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:48:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:53.104802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:53.194584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:53.203763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb158d-cce4-4e72-af99-48a669030202 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:53.236763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2f6c4-a0e9-4d78-a3a8-6aea97f61641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:53.237417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2f6c4-a0e9-4d78-a3a8-6aea97f61641 None admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:53.237947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2f6c4-a0e9-4d78-a3a8-6aea97f61641 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:53.237947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2f6c4-a0e9-4d78-a3a8-6aea97f61641 None admin] PATH_INFO: `/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/4c88695a14384a0b85bb20a0589692bb/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:53.239813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2f6c4-a0e9-4d78-a3a8-6aea97f61641 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcfaa7697faf49c29c93c4524c09369f, user_id=4c88695a14384a0b85bb20a0589692bb, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:53.241248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2f6c4-a0e9-4d78-a3a8-6aea97f61641 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:53.368781 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 640/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 1 17:48:53 2022] PUT /identity/v3/projects/dcfaa7697faf49c29c93c4524c09369f/users/4c88695a14384a0b85bb20a0589692bb/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 17:48:55.251235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72247082-9f17-467a-9942-e75092299b8a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:55.251755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72247082-9f17-467a-9942-e75092299b8a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:55.251862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72247082-9f17-467a-9942-e75092299b8a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:55.253602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 634/1274] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:55.258409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:55.260190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:55.260190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:55.278578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:55.280460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:55.349778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 641/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:55.621434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72247082-9f17-467a-9942-e75092299b8a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:55.721212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72247082-9f17-467a-9942-e75092299b8a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:55.738004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:55.754161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:55.754776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:55.755052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:55.755319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:55.765591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 17:48:55.770424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:55.771967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:55.779046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 635/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 1 17:48:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:55.779942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:55.856866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:55.874598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1ddaa-d12c-4771-91c9-f0c591e43b07 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:55.907903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:55.908530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:55.908809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:55.909079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:55.909740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:55.910039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:55.911379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:55.920011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 642/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 1 17:48:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:55.926474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:56.026982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960845e5-37bf-41af-915c-c266815e4ea5 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:56.060909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:56.061526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:56.061803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:56.062074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:56.071879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:56.073406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:56.088394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:56.089622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:56.090564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:56.090564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:56.090564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:56.090564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 636/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:56.095629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:56.194387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098c4da-5f41-4233-9700-089ff7b2f018 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:56.227172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:56.227769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:56.228047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:56.229024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:56.235783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:56.235783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:56.235783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:56.242755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 643/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:56.249002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:56.280856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:56.353113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3912d-b532-4991-bacf-0b8eed0fe73d None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:56.387627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:56.388256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:56.388589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:56.388863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:56.396601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:56.398026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 17:48:56.409788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:56.410816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:48:56.412380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 637/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:56.417541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:56.421425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:56.512918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe5592-dc4c-4be3-a94b-91177e521de0 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:56.547945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:56.548617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:56.548895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:56.549169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:56.549990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:56.550278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:56.551707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:56.568795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 644/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 1 17:48:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:56.575107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:56.685412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:56.695184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024de5e2-0e84-43df-a1b9-0681f5477d34 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:56.729312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:56.729907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:56.730184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:56.730452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:56.797654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:56.799035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:56.887310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:56.897448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 638/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 1 17:48:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 17:48:56.904046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:56.966434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:57.006686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a8ea8-8c81-4dfe-8df4-0d5298d2afb5 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:57.041611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:57.042223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:57.042498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:57.042764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:57.052636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:57.054017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 17:48:57.069352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:57.070534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 17:48:57.071703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 17:48:57.071703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 17:48:57.071703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:57.071703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 645/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:57.077287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:57.176333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2a299-d1ec-4ca9-ae0d-21548780a752 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:57.210593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:57.211191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:57.211472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:57.211745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:57.212410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:57.212750 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:57.214104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:57.225172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 639/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 1 17:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:57.230927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:57.331271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef1f1-68d0-4fe1-911d-ce3418ad8af1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:57.344948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:57.363945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:57.364594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:57.364871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:57.365141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:57.372107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:57.373548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 17:48:57.384520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application with Lock( Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:48:57.386107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: ERROR keystone.server.flask.application Sep 01 17:48:57.387089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 646/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 1 17:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:48:57.392017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:57.442363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:57.489773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c7324-7b7a-4cbe-a7aa-8c3d8ab21d15 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:57.523516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:57.524103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:57.524403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:57.524671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:57.525394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:57.525765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:57.527188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958f9cc-a7ce-4b60-91e4-85772c74b30e None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:57.538866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 640/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 1 17:48:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:57.545796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:48:57.643431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5b7d-1c91-4615-9a52-d9f43c47c40a None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:48:57.676075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:48:57.676693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:57.676967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:57.677233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:57.677947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] RBAC: Adding query filter params (role.id=19e3032b82e342bc93a63cb13b8b7478, scope.project.id=dcfaa7697faf49c29c93c4524c09369f, user.id=4c88695a14384a0b85bb20a0589692bb) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:48:57.678251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:48:57.679645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ffb77-865f-49b3-85d0-f6763a7ac745 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:48:57.685870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 647/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 1 17:48:57 2022] GET /identity/v3/role_assignments?user.id=4c88695a14384a0b85bb20a0589692bb&scope.project.id=dcfaa7697faf49c29c93c4524c09369f&role.id=19e3032b82e342bc93a63cb13b8b7478 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:48:59.624068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0903-beef-4f32-b5b6-b4ca3331de8a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:59.624677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0903-beef-4f32-b5b6-b4ca3331de8a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:59.624773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0903-beef-4f32-b5b6-b4ca3331de8a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:59.626080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 641/1288] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:48:59.631166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:48:59.631569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:48:59.631755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:48:59.649878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:48:59.651560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:48:59.682015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:48:59.725819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 648/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:48:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:48:59.992990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0903-beef-4f32-b5b6-b4ca3331de8a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:00.095859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0903-beef-4f32-b5b6-b4ca3331de8a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:00.106433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:00.130308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:00.130933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:00.131210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:00.131478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:00.132058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:00.133486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:49:00.143594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:49:00.144892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:00.146001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 642/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:48:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:49:00.151604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:00.249773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76352c78-47e4-4122-9830-0082f0ee41bb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:00.283922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:00.284553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:00.284831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:00.285103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:00.285813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:49:00.286110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:00.287447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766eb92d-b646-4145-8f22-3748060ec437 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:00.292587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 649/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 1 17:49:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:49:00.298421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:00.399630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76332e-9abf-409b-93f4-0f0d7be64d1c None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:00.434474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:00.438859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:00.438859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:00.438859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] PATH_INFO: `/v3/services` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:00.438859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] RBAC: Adding query filter params (type=placement) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:49:00.438859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:00.438859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cf080-3469-4487-add8-7eec1a3b30af None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:00.442518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 643/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 1 17:49:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:49:02.383482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f996fa-5871-4961-bce3-37222c0db376 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:02.384026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f996fa-5871-4961-bce3-37222c0db376 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:02.384026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f996fa-5871-4961-bce3-37222c0db376 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:02.386034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 650/1293] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:49:02.391130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97f4ea-4cd5-4445-8665-833cc6eaf066 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:02.391551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97f4ea-4cd5-4445-8665-833cc6eaf066 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:02.391681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97f4ea-4cd5-4445-8665-833cc6eaf066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:02.409927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-cb97f4ea-4cd5-4445-8665-833cc6eaf066 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:49:02.411102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cb97f4ea-4cd5-4445-8665-833cc6eaf066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:02.440928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:02.478181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 644/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:49:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:49:02.756364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f996fa-5871-4961-bce3-37222c0db376 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:02.787335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:02.855330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f996fa-5871-4961-bce3-37222c0db376 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:02.888522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34768ee-9374-4a95-9a50-f19edfca9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:02.889113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34768ee-9374-4a95-9a50-f19edfca9f62 None admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:02.889388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34768ee-9374-4a95-9a50-f19edfca9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:02.889656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34768ee-9374-4a95-9a50-f19edfca9f62 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:02.890308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34768ee-9374-4a95-9a50-f19edfca9f62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:02.891767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34768ee-9374-4a95-9a50-f19edfca9f62 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:03.008676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 651/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 1 17:49:02 2022] POST /identity/v3/services => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:49:05.041460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6449e-ade3-4aaf-95fc-ce7fef3ddd7e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:05.042024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6449e-ade3-4aaf-95fc-ce7fef3ddd7e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:05.042024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6449e-ade3-4aaf-95fc-ce7fef3ddd7e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:05.043355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 645/1296] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:49:05.048496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:05.048893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:05.049047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:05.067976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:49:05.069207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:05.140334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 652/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:49:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:49:05.409468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6449e-ade3-4aaf-95fc-ce7fef3ddd7e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:05.511403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6449e-ade3-4aaf-95fc-ce7fef3ddd7e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:05.544870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:05.545492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:05.545770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:05.546039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:05.546616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:05.547971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:49:05.557483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:49:05.558841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:05.559999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 646/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:49:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:49:05.564659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:05.583351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:05.662875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f909e4-7b34-4a05-b994-69b87fc02982 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:05.698869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:05.699475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:05.699753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:05.700026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:05.700686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:49:05.700988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:05.702334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:05.707406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 653/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 1 17:49:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:49:05.714346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:05.745401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:05.812973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf3aae-be44-4214-9a6e-de1956db9d86 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:05.846832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:05.847473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:05.847780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:05.848092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:05.848871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] RBAC: Adding query filter params (interface=public, service_id=5764a0cdd5d24ea19306066b0be7ae4b, region_id=RegionOne) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:49:05.849229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:05.850617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a07fb7-4f0c-4273-9d43-7920f66c3159 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:05.855765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 647/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 1 17:49:05 2022] GET /identity/v3/endpoints?service_id=5764a0cdd5d24ea19306066b0be7ae4b&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:49:05.861466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:05.943948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:05.961912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48085a95-7c31-4654-8fb2-d36d7ec1aeb1 None admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:05.996088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762aafd-a895-4422-b38d-6dbbea47edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:05.996710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762aafd-a895-4422-b38d-6dbbea47edb2 None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:05.996986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762aafd-a895-4422-b38d-6dbbea47edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:05.997256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762aafd-a895-4422-b38d-6dbbea47edb2 None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:05.997847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762aafd-a895-4422-b38d-6dbbea47edb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:05.999172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762aafd-a895-4422-b38d-6dbbea47edb2 None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:06.005756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 654/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 1 17:49:05 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:49:07.913010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d874-2a51-4c89-b360-de05001a09ef None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:07.913545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d874-2a51-4c89-b360-de05001a09ef None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:07.913545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d874-2a51-4c89-b360-de05001a09ef None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:07.914935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 648/1302] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:49:07.920753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:07.921128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:07.921361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:07.939300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:49:07.940522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:49:08.006678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 655/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:49:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 17:49:08.282578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d874-2a51-4c89-b360-de05001a09ef None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:08.377566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:08.384169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d874-2a51-4c89-b360-de05001a09ef None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:08.418038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:08.418656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:08.420748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:08.420748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:08.420748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:08.421361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 17:49:08.430056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 17:49:08.431616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application with Lock( Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self._enter() Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application return self.creator() Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application created_value = creator( Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: ERROR keystone.server.flask.application Sep 01 17:49:08.432619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 649/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 1 17:49:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 17:49:08.437965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:08.513237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:08.536957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ccb2f-e12a-4333-898f-85eb884d7248 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:08.571611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:08.572808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:08.573081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:08.573350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] PATH_INFO: `/v3/services` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:08.573978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 17:49:08.574279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:08.575618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ade16b-810f-45f9-9c23-99de2b2cfebb None admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:08.580687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 656/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 1 17:49:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 17:49:08.586657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:49:08.693896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c2014-c46d-4792-9c43-6810ff08c615 None admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:49:08.727767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae96e12-a43a-4ee3-a2ef-60d1c56c9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:49:08.728932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96e12-a43a-4ee3-a2ef-60d1c56c9090 None admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:49:08.729204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96e12-a43a-4ee3-a2ef-60d1c56c9090 None admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:49:08.729470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96e12-a43a-4ee3-a2ef-60d1c56c9090 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:49:08.730128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae96e12-a43a-4ee3-a2ef-60d1c56c9090 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:49:08.731491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae96e12-a43a-4ee3-a2ef-60d1c56c9090 None admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:49:08.831332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 650/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 17:49:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 17:49:18.578286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=30 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:49:18.826795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:42.211880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e83b8-ce45-4077-a166-f3ca55784785 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:42.212881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e83b8-ce45-4077-a166-f3ca55784785 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:42.213363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e83b8-ce45-4077-a166-f3ca55784785 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:42.220404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416512b-5261-47ee-80c3-4cc70dda357b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:42.221270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416512b-5261-47ee-80c3-4cc70dda357b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:42.221761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416512b-5261-47ee-80c3-4cc70dda357b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:42.224866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 657/1307] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:55:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:42.299814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c416512b-5261-47ee-80c3-4cc70dda357b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:42.301442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c416512b-5261-47ee-80c3-4cc70dda357b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:42.432892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 651/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:55:42.727156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15362f-e796-4c62-a966-eb9b64be9074 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:42.727489 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15362f-e796-4c62-a966-eb9b64be9074 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:42.727706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15362f-e796-4c62-a966-eb9b64be9074 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:42.729123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 658/1309] 10.222.0.15 () {54 vars in 965 bytes} [Thu Sep 1 17:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:42.735196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc5a0-b8d3-4fcd-8123-4e01fc1b65c6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:42.735486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc5a0-b8d3-4fcd-8123-4e01fc1b65c6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:42.735858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc5a0-b8d3-4fcd-8123-4e01fc1b65c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:42.818971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-830cc5a0-b8d3-4fcd-8123-4e01fc1b65c6 None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:42.820417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-830cc5a0-b8d3-4fcd-8123-4e01fc1b65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:42.841747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:42.901180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 652/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Thu Sep 1 17:55:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:42.909986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15362f-e796-4c62-a966-eb9b64be9074 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:42.911796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3e15362f-e796-4c62-a966-eb9b64be9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:43.027082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15362f-e796-4c62-a966-eb9b64be9074 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:43.099616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:43.100963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:43.101241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:43.101509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:43.103104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:43.158186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:43.224300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:43.226588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65ad4f-b365-47a9-b03d-a46de1bbd3c4 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:43.247817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 659/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:45.525680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df1d77-8c25-4cc4-8b21-5bff8889b9cd None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:45.526215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df1d77-8c25-4cc4-8b21-5bff8889b9cd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:45.526445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df1d77-8c25-4cc4-8b21-5bff8889b9cd None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:45.528040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 653/1312] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:45.533007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552cfbc-c025-453f-898a-a4e6e867363a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:45.533408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552cfbc-c025-453f-898a-a4e6e867363a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:45.533552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552cfbc-c025-453f-898a-a4e6e867363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:45.631950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f552cfbc-c025-453f-898a-a4e6e867363a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:45.633649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f552cfbc-c025-453f-898a-a4e6e867363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:45.750388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 660/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:46.035351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df1d77-8c25-4cc4-8b21-5bff8889b9cd None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:46.036903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f1df1d77-8c25-4cc4-8b21-5bff8889b9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:46.146017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df1d77-8c25-4cc4-8b21-5bff8889b9cd None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:46.222377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:46.224968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:46.225668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:46.225940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:46.226207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:46.227834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:46.340615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:46.342055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8068-613f-4b5e-8398-5bd43be4062e service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:46.361158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 654/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:48.768728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8b201-f845-4c33-8a0d-f2d0ebdbdb88 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:48.769299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8b201-f845-4c33-8a0d-f2d0ebdbdb88 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:48.769299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8b201-f845-4c33-8a0d-f2d0ebdbdb88 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:48.770687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 661/1315] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:48.777067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d358e-8992-4619-891f-f2ba0abd4942 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:48.777464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d358e-8992-4619-891f-f2ba0abd4942 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:48.777621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d358e-8992-4619-891f-f2ba0abd4942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:48.843417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-649d358e-8992-4619-891f-f2ba0abd4942 None None] MFA Rules not processed for user `5ad5ace901534fc38bd2b9d0072d016b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:48.844657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-649d358e-8992-4619-891f-f2ba0abd4942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:48.896622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:48.942382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 655/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:55:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:49.227459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8b201-f845-4c33-8a0d-f2d0ebdbdb88 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:49.229031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a9e8b201-f845-4c33-8a0d-f2d0ebdbdb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:49.238883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:49.333732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8b201-f845-4c33-8a0d-f2d0ebdbdb88 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:49.406243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:49.406877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:49.407162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:49.407436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:49.409157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:49.470691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:49.515837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:49.517206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448f74a-2cb3-41a0-b8b5-709e9a473eaa service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:49.535304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 662/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:55:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:52.327375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867853b-1bd7-4214-8fab-4877e39607b3 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:52.327938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867853b-1bd7-4214-8fab-4877e39607b3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:52.327938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867853b-1bd7-4214-8fab-4877e39607b3 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:52.329287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 656/1318] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:52.334223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40f1c0-cb42-4606-a17f-70fd3636a3f5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:52.334595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40f1c0-cb42-4606-a17f-70fd3636a3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:52.334780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40f1c0-cb42-4606-a17f-70fd3636a3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:52.400185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ae40f1c0-cb42-4606-a17f-70fd3636a3f5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:52.401406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ae40f1c0-cb42-4606-a17f-70fd3636a3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:52.497157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 663/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:55:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:52.780205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867853b-1bd7-4214-8fab-4877e39607b3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:52.781755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c867853b-1bd7-4214-8fab-4877e39607b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:52.884047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867853b-1bd7-4214-8fab-4877e39607b3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:52.955632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:52.956225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:52.956525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:52.956793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:52.958363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:52.966105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:53.065690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:53.067032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8376809-0006-41c4-82dc-2ce6b0ddb346 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:53.085646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 657/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:55:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:56.065135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec400f5-1024-418d-9759-25b2a75931b1 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:56.065646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec400f5-1024-418d-9759-25b2a75931b1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:56.065646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec400f5-1024-418d-9759-25b2a75931b1 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:56.066931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 664/1321] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:56.072987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173528b-d4d0-4fc7-a5af-900d663553f5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:56.073396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173528b-d4d0-4fc7-a5af-900d663553f5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:56.073649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173528b-d4d0-4fc7-a5af-900d663553f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:56.139688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9173528b-d4d0-4fc7-a5af-900d663553f5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:56.140964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9173528b-d4d0-4fc7-a5af-900d663553f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:56.200864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:56.245499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 658/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:56.579519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec400f5-1024-418d-9759-25b2a75931b1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:56.579519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0ec400f5-1024-418d-9759-25b2a75931b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:56.579519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:56.666287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec400f5-1024-418d-9759-25b2a75931b1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:56.739174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:56.739782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:56.740058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:56.740367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:56.742159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:56.812975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:56.850424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:56.851726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a28a-6a61-426c-a599-5f145d9400e6 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:56.870321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 665/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:55:58.863383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258df565-e3f7-4d95-bb22-90b60f23cb13 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:58.863957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258df565-e3f7-4d95-bb22-90b60f23cb13 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:58.863957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258df565-e3f7-4d95-bb22-90b60f23cb13 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:58.865589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 659/1324] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:58.870714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87218-558d-4330-9ff7-f85fe885a492 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:58.871125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87218-558d-4330-9ff7-f85fe885a492 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:58.871263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87218-558d-4330-9ff7-f85fe885a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:58.938253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d8e87218-558d-4330-9ff7-f85fe885a492 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:58.939365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d8e87218-558d-4330-9ff7-f85fe885a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:59.041757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 666/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:59.362832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e9705-ad1d-40d1-8e35-bc15a4f2c4f0 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:59.363119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e9705-ad1d-40d1-8e35-bc15a4f2c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:59.363395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e9705-ad1d-40d1-8e35-bc15a4f2c4f0 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:59.364792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 660/1326] 10.222.0.15 () {54 vars in 965 bytes} [Thu Sep 1 17:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:55:59.371074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769094d-06a6-4eac-a529-8955c0691095 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:59.371464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769094d-06a6-4eac-a529-8955c0691095 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:59.371749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769094d-06a6-4eac-a529-8955c0691095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:59.405924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:59.452090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c769094d-06a6-4eac-a529-8955c0691095 None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:55:59.453311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c769094d-06a6-4eac-a529-8955c0691095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:59.537972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 667/1327] 10.222.0.15 () {58 vars in 1062 bytes} [Thu Sep 1 17:55:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:55:59.547082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e9705-ad1d-40d1-8e35-bc15a4f2c4f0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:55:59.548801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-976e9705-ad1d-40d1-8e35-bc15a4f2c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:59.654287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e9705-ad1d-40d1-8e35-bc15a4f2c4f0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:55:59.741911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:55:59.742520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:55:59.742798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:55:59.743068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:55:59.744701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:55:59.761658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:55:59.855335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:55:59.856728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b9432-21d9-4307-a0e8-225406d7a7c7 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:55:59.874635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 661/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:02.092012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33445a-f45f-43a9-be96-9cf7aeba74ec None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:02.092555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33445a-f45f-43a9-be96-9cf7aeba74ec None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:02.092646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33445a-f45f-43a9-be96-9cf7aeba74ec None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:02.093942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 668/1329] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:02.099086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feae51c-e773-4a4a-838f-8baaa643a53b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:02.099473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feae51c-e773-4a4a-838f-8baaa643a53b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:02.100096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feae51c-e773-4a4a-838f-8baaa643a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:02.179969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4feae51c-e773-4a4a-838f-8baaa643a53b None None] MFA Rules not processed for user `5ad5ace901534fc38bd2b9d0072d016b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:02.181237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4feae51c-e773-4a4a-838f-8baaa643a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:02.253973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:02.286623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 662/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:02.592547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33445a-f45f-43a9-be96-9cf7aeba74ec None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:02.594120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cd33445a-f45f-43a9-be96-9cf7aeba74ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:02.705020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33445a-f45f-43a9-be96-9cf7aeba74ec None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:02.726604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:02.783062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:02.783767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:02.784033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:02.784324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:02.785938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:02.901419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:02.902867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715770d-d2bc-4a78-9201-09fab90e26e1 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:02.921344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 669/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:05.404340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303f24a-a390-4fb4-9ffa-393bae7788f3 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:05.404885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303f24a-a390-4fb4-9ffa-393bae7788f3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:05.404885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303f24a-a390-4fb4-9ffa-393bae7788f3 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:05.406251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 663/1332] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:05.411304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402b134-c349-4646-859a-6f45932c0337 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:05.411697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402b134-c349-4646-859a-6f45932c0337 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:05.411872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402b134-c349-4646-859a-6f45932c0337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:05.498965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:05.501609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b402b134-c349-4646-859a-6f45932c0337 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:05.502925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b402b134-c349-4646-859a-6f45932c0337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:05.601026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 670/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:05.922228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303f24a-a390-4fb4-9ffa-393bae7788f3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:05.923765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6303f24a-a390-4fb4-9ffa-393bae7788f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:06.026338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303f24a-a390-4fb4-9ffa-393bae7788f3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:06.097862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:06.098461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:06.098740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:06.099008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:06.101166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:06.127922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:06.223109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:06.225847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b20144-8c30-4155-85b2-6d754ac22b65 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:06.245112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 664/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:08.257333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a9021-92ad-4b0a-8bb6-9c9533d346ac None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:08.257900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a9021-92ad-4b0a-8bb6-9c9533d346ac None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:08.258085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a9021-92ad-4b0a-8bb6-9c9533d346ac None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:08.259496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 671/1335] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:08.265188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d606be-f6ea-4a1b-b55f-ff734b879ee8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:08.265635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d606be-f6ea-4a1b-b55f-ff734b879ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:08.265731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d606be-f6ea-4a1b-b55f-ff734b879ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:08.341189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f2d606be-f6ea-4a1b-b55f-ff734b879ee8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:08.343256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f2d606be-f6ea-4a1b-b55f-ff734b879ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:08.432465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:08.469024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 665/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:08.770504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a9021-92ad-4b0a-8bb6-9c9533d346ac None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:08.772166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c35a9021-92ad-4b0a-8bb6-9c9533d346ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:08.874568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a9021-92ad-4b0a-8bb6-9c9533d346ac None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:08.902327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:08.949766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:08.950369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:08.950645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:08.950916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:08.952526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:09.059398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:09.060759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a8c34-b0b3-4d1c-a34d-42a968dc72e3 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:09.078770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 672/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:11.818850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c45f-9c69-492e-8696-01297c8c8369 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:11.819375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c45f-9c69-492e-8696-01297c8c8369 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:11.819473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c45f-9c69-492e-8696-01297c8c8369 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:11.820836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 666/1338] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:11.825941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d7659-d768-4e6c-a905-7e73bfdd8bc3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:11.826358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d7659-d768-4e6c-a905-7e73bfdd8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:11.826472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d7659-d768-4e6c-a905-7e73bfdd8bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:11.891693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-771d7659-d768-4e6c-a905-7e73bfdd8bc3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:11.892908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-771d7659-d768-4e6c-a905-7e73bfdd8bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:11.901344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:11.988675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 673/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:12.271583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5c45f-9c69-492e-8696-01297c8c8369 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:12.273422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-65a5c45f-9c69-492e-8696-01297c8c8369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:12.375150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5c45f-9c69-492e-8696-01297c8c8369 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:12.446753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:12.447348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:12.447625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:12.447893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:12.449576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:12.484509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:12.558367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:12.559709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c70a7-8f7e-49e7-ad43-061637e69781 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:12.579320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 667/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:16.852050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a010df-9bcc-45ec-bc64-93c007450ed6 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:16.852616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a010df-9bcc-45ec-bc64-93c007450ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:16.852616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a010df-9bcc-45ec-bc64-93c007450ed6 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:16.853972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 674/1341] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:16.859188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d163f-a300-43c7-8c4b-b94e81cdcb2e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:16.859628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d163f-a300-43c7-8c4b-b94e81cdcb2e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:16.859741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d163f-a300-43c7-8c4b-b94e81cdcb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:16.926450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8e7d163f-a300-43c7-8c4b-b94e81cdcb2e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:16.929888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8e7d163f-a300-43c7-8c4b-b94e81cdcb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:17.016766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:17.028185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 668/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:56:17.308548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a010df-9bcc-45ec-bc64-93c007450ed6 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:17.310078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-48a010df-9bcc-45ec-bc64-93c007450ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:17.415813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a010df-9bcc-45ec-bc64-93c007450ed6 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:17.454793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:17.492028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:17.492672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:17.492952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:17.493225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:17.494808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:17.608041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:17.609406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c113-6ced-44f1-bbc8-5b081a3569c9 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:17.628009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 675/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:20.465662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902afd0-5cdb-4763-bde5-c573b50cde0e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:20.466186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902afd0-5cdb-4763-bde5-c573b50cde0e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:20.466301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902afd0-5cdb-4763-bde5-c573b50cde0e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:20.467569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 669/1344] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:20.472924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91d425-606e-438d-832f-b6eb6b66ca29 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:20.473335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91d425-606e-438d-832f-b6eb6b66ca29 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:20.473475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91d425-606e-438d-832f-b6eb6b66ca29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:20.539380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-eb91d425-606e-438d-832f-b6eb6b66ca29 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:20.540602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-eb91d425-606e-438d-832f-b6eb6b66ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:20.558446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:20.643049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 676/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:20.925727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902afd0-5cdb-4763-bde5-c573b50cde0e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:20.927867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c902afd0-5cdb-4763-bde5-c573b50cde0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:21.039194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902afd0-5cdb-4763-bde5-c573b50cde0e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:21.112173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:21.112801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:21.113076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:21.113342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:21.115690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:21.159038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:21.229310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:21.230764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35045c43-c4a9-41da-a0e6-4ff688c1272d service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:21.250210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 670/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:25.625848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a53fe2-42b1-4ece-b7a8-63905e479700 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:25.626325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a53fe2-42b1-4ece-b7a8-63905e479700 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:25.626452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a53fe2-42b1-4ece-b7a8-63905e479700 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:25.627737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 677/1347] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:25.633995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d30eb-edae-4a3a-bd95-5faba0ff1393 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:25.634525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d30eb-edae-4a3a-bd95-5faba0ff1393 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:25.634525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d30eb-edae-4a3a-bd95-5faba0ff1393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:25.705551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7c8d30eb-edae-4a3a-bd95-5faba0ff1393 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:25.706773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7c8d30eb-edae-4a3a-bd95-5faba0ff1393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:25.809115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:25.811705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 671/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:26.101240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a53fe2-42b1-4ece-b7a8-63905e479700 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:26.102777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-03a53fe2-42b1-4ece-b7a8-63905e479700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:26.213062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a53fe2-42b1-4ece-b7a8-63905e479700 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:26.262918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:26.290255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:26.290864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:26.291138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:26.291402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:26.293012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:26.403063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:26.404515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057a8b7-b305-4e4c-a392-d23d54f765ee service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:26.426844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 678/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:56:31.352933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6bf4d-f9c4-4710-9f9f-2a5aa35c8432 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:31.353446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6bf4d-f9c4-4710-9f9f-2a5aa35c8432 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:31.353537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6bf4d-f9c4-4710-9f9f-2a5aa35c8432 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:31.355137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 672/1350] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 17:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:31.360179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc5f02-3877-49e3-ac07-0dbe692f342f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:31.360823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc5f02-3877-49e3-ac07-0dbe692f342f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:31.360975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc5f02-3877-49e3-ac07-0dbe692f342f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:31.457748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7ccc5f02-3877-49e3-ac07-0dbe692f342f None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:31.460049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7ccc5f02-3877-49e3-ac07-0dbe692f342f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:31.489808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:31.667627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 679/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:35.854060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce514e9c-4bbe-4790-8ca1-173e35f2d4df None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:35.854597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce514e9c-4bbe-4790-8ca1-173e35f2d4df None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:35.855019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce514e9c-4bbe-4790-8ca1-173e35f2d4df None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:35.857169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 673/1352] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 17:56:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:35.862363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b2ddb-8b42-496c-81c1-7bfbfaf1b3f6 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:35.862801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b2ddb-8b42-496c-81c1-7bfbfaf1b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:35.863110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b2ddb-8b42-496c-81c1-7bfbfaf1b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:35.943142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-018b2ddb-8b42-496c-81c1-7bfbfaf1b3f6 None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:35.944534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-018b2ddb-8b42-496c-81c1-7bfbfaf1b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:36.018304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:36.057169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 680/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 17:56:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:37.480671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ffe23-c7b8-461c-9930-25754676fc36 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:37.481191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ffe23-c7b8-461c-9930-25754676fc36 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:37.481191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ffe23-c7b8-461c-9930-25754676fc36 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:37.482626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 674/1354] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 17:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:37.487707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b52dc-c698-42d6-86b5-64d4364a0b47 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:37.488129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b52dc-c698-42d6-86b5-64d4364a0b47 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:37.488434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b52dc-c698-42d6-86b5-64d4364a0b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:37.566713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-dc4b52dc-c698-42d6-86b5-64d4364a0b47 None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:37.568588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dc4b52dc-c698-42d6-86b5-64d4364a0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:37.680840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 681/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:40.999283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dc09e-47b8-4eda-8e9c-4754d410283e None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:41.000138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dc09e-47b8-4eda-8e9c-4754d410283e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:41.000589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dc09e-47b8-4eda-8e9c-4754d410283e None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:41.002336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 675/1356] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:41.007963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b9d11-82e5-4dc7-8948-e30a66066c30 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:41.008519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b9d11-82e5-4dc7-8948-e30a66066c30 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:41.008616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b9d11-82e5-4dc7-8948-e30a66066c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:41.056590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:41.088058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bc0b9d11-82e5-4dc7-8948-e30a66066c30 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:41.091232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bc0b9d11-82e5-4dc7-8948-e30a66066c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:41.233414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 682/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:41.538860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb25d1-d9cb-49fb-b3b4-da81e4a02df0 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:41.541872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb25d1-d9cb-49fb-b3b4-da81e4a02df0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:41.541872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb25d1-d9cb-49fb-b3b4-da81e4a02df0 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:41.541872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 676/1358] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 1 17:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:41.549676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758c75-cfe7-46e0-9e80-059daf3cab73 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:41.549970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758c75-cfe7-46e0-9e80-059daf3cab73 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:41.550250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758c75-cfe7-46e0-9e80-059daf3cab73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:41.640303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ea758c75-cfe7-46e0-9e80-059daf3cab73 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:41.641668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ea758c75-cfe7-46e0-9e80-059daf3cab73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:41.704398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:41.778198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 683/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 1 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:41.786250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb25d1-d9cb-49fb-b3b4-da81e4a02df0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:41.788206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-63fb25d1-d9cb-49fb-b3b4-da81e4a02df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:41.946570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb25d1-d9cb-49fb-b3b4-da81e4a02df0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:42.030633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:42.066310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:42.067633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:42.068058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:42.068460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:42.070275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:42.165325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:42.167083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca13c94-15b7-4dc7-9223-feb2d96746cc service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:42.186163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 677/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:44.317554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d4cb1-d8d8-4db2-a99f-1f93a822a9ae None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:44.317979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d4cb1-d8d8-4db2-a99f-1f93a822a9ae None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:44.318092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d4cb1-d8d8-4db2-a99f-1f93a822a9ae None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:44.319464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 684/1361] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:44.324729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba93e-8e97-4574-b02a-ac1b5cc05f15 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:44.325165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba93e-8e97-4574-b02a-ac1b5cc05f15 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:44.325269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba93e-8e97-4574-b02a-ac1b5cc05f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:44.403279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bf1ba93e-8e97-4574-b02a-ac1b5cc05f15 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:44.404737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bf1ba93e-8e97-4574-b02a-ac1b5cc05f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:44.447113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:44.510200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 678/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:44.809275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d4cb1-d8d8-4db2-a99f-1f93a822a9ae None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:44.810876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-715d4cb1-d8d8-4db2-a99f-1f93a822a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:45.006554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d4cb1-d8d8-4db2-a99f-1f93a822a9ae None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:45.016931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:45.133888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:45.134452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:45.134801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:45.135246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:45.136958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:45.240076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:45.242006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9922-0157-44c8-993f-0c4a7c5643e5 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:45.263968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 685/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:47.669588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2186-2967-4a8e-9a71-a776aa7c91f2 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:47.670178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2186-2967-4a8e-9a71-a776aa7c91f2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:47.670178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2186-2967-4a8e-9a71-a776aa7c91f2 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:47.671468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 679/1364] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:47.677078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8931e-ccc2-4709-b4de-b4543387b6c0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:47.677567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8931e-ccc2-4709-b4de-b4543387b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:47.677849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8931e-ccc2-4709-b4de-b4543387b6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:47.695099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:47.755101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e4f8931e-ccc2-4709-b4de-b4543387b6c0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:47.756333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e4f8931e-ccc2-4709-b4de-b4543387b6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:47.861526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 686/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:48.160195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561279-8856-4e29-9848-830e003b480f None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:48.160593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561279-8856-4e29-9848-830e003b480f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:48.160883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7561279-8856-4e29-9848-830e003b480f None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:48.162685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 680/1366] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 1 17:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:48.168723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7132e-4f04-4424-ac1a-62aa2948662c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:48.169012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7132e-4f04-4424-ac1a-62aa2948662c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:48.169304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7132e-4f04-4424-ac1a-62aa2948662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:48.256885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fce7132e-4f04-4424-ac1a-62aa2948662c None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:48.258423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fce7132e-4f04-4424-ac1a-62aa2948662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:48.296204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:48.383826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 687/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 1 17:56:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:48.391823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7561279-8856-4e29-9848-830e003b480f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:48.394520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e7561279-8856-4e29-9848-830e003b480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:48.560758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7561279-8856-4e29-9848-830e003b480f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:48.586025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:48.687311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:48.687970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:48.688229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:48.688503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:48.690211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:48.792561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:48.794163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74125e3b-7ef2-4009-8fc6-61371ff9a583 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:48.813270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 681/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:49.346457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733584a-8b48-4238-9491-879564705a02 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:49.346976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733584a-8b48-4238-9491-879564705a02 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:49.347059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733584a-8b48-4238-9491-879564705a02 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:49.348475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 688/1369] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:49.353831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e288b6-04e9-401b-b1be-21bf2a500258 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:49.354125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e288b6-04e9-401b-b1be-21bf2a500258 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:49.354405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e288b6-04e9-401b-b1be-21bf2a500258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:49.387950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:49.426844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-43e288b6-04e9-401b-b1be-21bf2a500258 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:49.428074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-43e288b6-04e9-401b-b1be-21bf2a500258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:49.534698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 682/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:50.124806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb04fb-f5ab-475d-8f03-fc1bcdbb6d5f None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:50.125033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb04fb-f5ab-475d-8f03-fc1bcdbb6d5f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:50.125611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb04fb-f5ab-475d-8f03-fc1bcdbb6d5f None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:50.126757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 689/1371] 10.222.0.15 () {54 vars in 964 bytes} [Thu Sep 1 17:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:50.131934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ccdd3-eb84-4f83-ba80-b67972eef076 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:50.132424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ccdd3-eb84-4f83-ba80-b67972eef076 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:50.132572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ccdd3-eb84-4f83-ba80-b67972eef076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:50.216346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7a9ccdd3-eb84-4f83-ba80-b67972eef076 None None] MFA Rules not processed for user `9c32115f24264e738c432f700a2f7dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:50.217747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7a9ccdd3-eb84-4f83-ba80-b67972eef076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:50.273146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:50.305103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 683/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Thu Sep 1 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:50.313028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb04fb-f5ab-475d-8f03-fc1bcdbb6d5f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:50.314659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-26eb04fb-f5ab-475d-8f03-fc1bcdbb6d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:50.422891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb04fb-f5ab-475d-8f03-fc1bcdbb6d5f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:50.469000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:50.506471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:50.508100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:50.508100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:50.508298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:50.509944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:50.627175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:50.628702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b37411-04f8-402f-a61d-fa424807c087 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:50.649568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 690/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:50.952259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a858472-2ab1-4481-8ca1-f8e33403d495 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:50.952597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a858472-2ab1-4481-8ca1-f8e33403d495 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:50.952876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a858472-2ab1-4481-8ca1-f8e33403d495 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:50.954261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 684/1374] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:50.960150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d2c6-2b52-4ca2-b747-88b58bdd3a9e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:50.960494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d2c6-2b52-4ca2-b747-88b58bdd3a9e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:50.960772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d2c6-2b52-4ca2-b747-88b58bdd3a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:51.026832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3b84d2c6-2b52-4ca2-b747-88b58bdd3a9e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:51.028335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3b84d2c6-2b52-4ca2-b747-88b58bdd3a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:51.048853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:51.133648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 691/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:51.425608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a858472-2ab1-4481-8ca1-f8e33403d495 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:51.427865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6a858472-2ab1-4481-8ca1-f8e33403d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:51.577487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a858472-2ab1-4481-8ca1-f8e33403d495 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:51.628336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:51.691997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:51.692634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:51.692985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:51.693327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:51.694998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:51.786478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:51.788081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93f847-2a45-491c-9bbe-453b9a2ab36a service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:51.806649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 685/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:53.916901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10356a-1740-4bdc-9dd4-4cd722e93beb None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:53.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10356a-1740-4bdc-9dd4-4cd722e93beb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:53.917504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10356a-1740-4bdc-9dd4-4cd722e93beb None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:53.918950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 692/1377] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:53.924251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2728-3c38-469d-b3f7-5114e127d7f1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:53.924724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2728-3c38-469d-b3f7-5114e127d7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:53.924821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2728-3c38-469d-b3f7-5114e127d7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:53.993648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-98fa2728-3c38-469d-b3f7-5114e127d7f1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:53.994883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-98fa2728-3c38-469d-b3f7-5114e127d7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:54.002599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:54.097471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 686/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:54.406188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10356a-1740-4bdc-9dd4-4cd722e93beb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:54.408241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1d10356a-1740-4bdc-9dd4-4cd722e93beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:54.579799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10356a-1740-4bdc-9dd4-4cd722e93beb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:54.588084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:54.706119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:54.706735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:54.707042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:54.707856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:54.709564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:54.810336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:54.812324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f0505-524a-49f1-a428-5f871937bc41 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:54.832760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 693/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:56:56.954579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f43c7-f554-4e72-a8ef-14d01bc3ba60 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:56.955102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f43c7-f554-4e72-a8ef-14d01bc3ba60 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:56.955102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f43c7-f554-4e72-a8ef-14d01bc3ba60 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:56.961124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 687/1380] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:56:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:56:56.962642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c095c7-388b-4b51-9f1d-13f6a8af67a0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:56.963084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c095c7-388b-4b51-9f1d-13f6a8af67a0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:56.963432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c095c7-388b-4b51-9f1d-13f6a8af67a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:56.983988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:57.049798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-88c095c7-388b-4b51-9f1d-13f6a8af67a0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:56:57.051185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-88c095c7-388b-4b51-9f1d-13f6a8af67a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:57.160364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 694/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:56:57.459358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f43c7-f554-4e72-a8ef-14d01bc3ba60 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:56:57.460888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-096f43c7-f554-4e72-a8ef-14d01bc3ba60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:57.620369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:57.630856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f43c7-f554-4e72-a8ef-14d01bc3ba60 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:56:57.754122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:56:57.754726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:56:57.754997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:56:57.755259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:56:57.756857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:56:57.850264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:56:57.852083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4429e-430e-449e-a9e4-a87584d6d187 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:56:57.862969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:56:57.875038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 688/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:00.056681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adff56-f42e-4481-bcfd-4d8423db77ee None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:00.057383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adff56-f42e-4481-bcfd-4d8423db77ee None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:00.057383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adff56-f42e-4481-bcfd-4d8423db77ee None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:00.058603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 695/1383] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:00.063930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244f926-bdc5-4653-b5e0-b99524588d73 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:00.064500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244f926-bdc5-4653-b5e0-b99524588d73 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:00.064500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244f926-bdc5-4653-b5e0-b99524588d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:00.132250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e244f926-bdc5-4653-b5e0-b99524588d73 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:00.133443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e244f926-bdc5-4653-b5e0-b99524588d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:00.240212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 689/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:00.531473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adff56-f42e-4481-bcfd-4d8423db77ee None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:00.533461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-71adff56-f42e-4481-bcfd-4d8423db77ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:00.643183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:00.713533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adff56-f42e-4481-bcfd-4d8423db77ee None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:00.828704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:00.829192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:00.829472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:00.829747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:00.831367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:00.885740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:00.933813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:00.935865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c33d8-6952-4534-8346-529636ffb009 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:00.959782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 696/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:02.977711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a33b7-58fb-48e0-98a8-14b10ae9da80 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:02.978377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a33b7-58fb-48e0-98a8-14b10ae9da80 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:02.978377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a33b7-58fb-48e0-98a8-14b10ae9da80 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:02.979613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 690/1386] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:02.984824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e043cf3-d69a-44a6-b33c-6f2f5ca6406a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:02.985312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e043cf3-d69a-44a6-b33c-6f2f5ca6406a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:02.985312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e043cf3-d69a-44a6-b33c-6f2f5ca6406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:03.057069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-5e043cf3-d69a-44a6-b33c-6f2f5ca6406a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:03.058309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5e043cf3-d69a-44a6-b33c-6f2f5ca6406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:03.164362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 697/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:03.455318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a33b7-58fb-48e0-98a8-14b10ae9da80 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:03.457384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-635a33b7-58fb-48e0-98a8-14b10ae9da80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:03.544484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:03.622157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a33b7-58fb-48e0-98a8-14b10ae9da80 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:03.752589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:03.753357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:03.754367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:03.754721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:03.757045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:03.802997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:03.858231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:03.859838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3222e-1c93-4e10-88b6-1886ac1f4dd2 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:03.879633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 691/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:05.949765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669da17-9aa5-400e-8fc6-50fa9564e22c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:05.950336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669da17-9aa5-400e-8fc6-50fa9564e22c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:05.950336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669da17-9aa5-400e-8fc6-50fa9564e22c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:05.951680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 698/1389] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:05.956885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d15ab8-1004-4472-83e2-9560ab0654dd None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:05.957373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d15ab8-1004-4472-83e2-9560ab0654dd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:05.957561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d15ab8-1004-4472-83e2-9560ab0654dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:06.024849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-08d15ab8-1004-4472-83e2-9560ab0654dd None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:06.026063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-08d15ab8-1004-4472-83e2-9560ab0654dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:06.126244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 692/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:06.278556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f70a1-7695-46ab-af94-d70ad3af28fe None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:06.278838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f70a1-7695-46ab-af94-d70ad3af28fe None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:06.279110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f70a1-7695-46ab-af94-d70ad3af28fe None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:06.280482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 699/1391] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:06.285830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ec9b7-b707-4470-8040-5a50003d25f2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:06.286113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ec9b7-b707-4470-8040-5a50003d25f2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:06.286386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ec9b7-b707-4470-8040-5a50003d25f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:06.303248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:06.354780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-654ec9b7-b707-4470-8040-5a50003d25f2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:06.355986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-654ec9b7-b707-4470-8040-5a50003d25f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:06.418769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f70a1-7695-46ab-af94-d70ad3af28fe None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:06.422616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2c2f70a1-7695-46ab-af94-d70ad3af28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:06.454749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:06.459312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 693/1392] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:06.571599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f70a1-7695-46ab-af94-d70ad3af28fe None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:06.673168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:06.684904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:06.685519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:06.685813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:06.686085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:06.687670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:06.762308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c4d0-d02e-4d6a-b68e-ffea6db08370 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:06.762593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c4d0-d02e-4d6a-b68e-ffea6db08370 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:06.762864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c4d0-d02e-4d6a-b68e-ffea6db08370 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:06.764215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 694/1393] 10.222.0.15 () {54 vars in 967 bytes} [Thu Sep 1 17:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:57:06.769070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23f88a-f3b4-47ca-bcd2-2f0dab91e049 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:06.769928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23f88a-f3b4-47ca-bcd2-2f0dab91e049 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:06.770200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23f88a-f3b4-47ca-bcd2-2f0dab91e049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:06.782976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:06.784599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:06.804016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 700/1394] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:57:06.848448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4a23f88a-f3b4-47ca-bcd2-2f0dab91e049 None None] MFA Rules not processed for user `dbb0ec082bdb45b0bb2c59cbad440d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:06.849727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4a23f88a-f3b4-47ca-bcd2-2f0dab91e049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:06.894444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:06.937509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 695/1395] 10.222.0.15 () {58 vars in 1064 bytes} [Thu Sep 1 17:57:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:57:06.945129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:06.946391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:07.032058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43de5-ea5b-4b21-8a59-0286313c0ef8 service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:07.079257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:07.117429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:07.117959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:07.118232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:07.118498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:07.120464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:07.242249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:07.243734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bb763-2db2-4e47-8a4c-4d8dd118009d service glance] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:07.266261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 701/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:08.302814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9fb8-8d13-405c-b768-bcd6f967e1fc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:08.303324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9fb8-8d13-405c-b768-bcd6f967e1fc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:08.303421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9fb8-8d13-405c-b768-bcd6f967e1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:08.386942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3e6b9fb8-8d13-405c-b768-bcd6f967e1fc None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:08.388522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3e6b9fb8-8d13-405c-b768-bcd6f967e1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:08.506908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 696/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:57:08.560642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477da4d-3117-4e5e-9892-eda8a7f6cbe8 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:08.561160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477da4d-3117-4e5e-9892-eda8a7f6cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:08.561341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477da4d-3117-4e5e-9892-eda8a7f6cbe8 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:08.562763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 702/1398] 10.222.0.15 () {54 vars in 943 bytes} [Thu Sep 1 17:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:08.570063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ab4d-df82-41b9-ac58-5df8e89222fa None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:08.570381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ab4d-df82-41b9-ac58-5df8e89222fa None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:08.570653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ab4d-df82-41b9-ac58-5df8e89222fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:08.629088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:08.647806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-17e8ab4d-df82-41b9-ac58-5df8e89222fa None None] MFA Rules not processed for user `42c1211f644d45bb8979455c4f28284a`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:08.649301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-17e8ab4d-df82-41b9-ac58-5df8e89222fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:08.758037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 697/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Thu Sep 1 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:08.767106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477da4d-3117-4e5e-9892-eda8a7f6cbe8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:08.768645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b477da4d-3117-4e5e-9892-eda8a7f6cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:08.845267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:08.921321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477da4d-3117-4e5e-9892-eda8a7f6cbe8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:09.047045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:09.047602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:09.047877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:09.048146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:09.049940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:09.109178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:09.129307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dacadb-5991-4cc0-b542-c73c32a079bf None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:09.129653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dacadb-5991-4cc0-b542-c73c32a079bf None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:09.129935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dacadb-5991-4cc0-b542-c73c32a079bf None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:09.131766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 698/1400] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 17:57:09.134973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:09.137525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:09.142800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e274db-784c-4e0b-9936-1c07f23f9903 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:09.142800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e274db-784c-4e0b-9936-1c07f23f9903 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:09.142800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e274db-784c-4e0b-9936-1c07f23f9903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:09.146356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 703/1401] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 17:57:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 17:57:09.212539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d5e274db-784c-4e0b-9936-1c07f23f9903 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:09.213804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d5e274db-784c-4e0b-9936-1c07f23f9903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:09.278665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:09.323044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 699/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:57:09.627396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:09.629508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:09.727952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8984b-2af5-489b-bb59-362e5a0fab4f service swift] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:09.844903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:09.857240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:09.857853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:09.859054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:09.859054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:09.859982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:09.968633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:09.970215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102e828-175b-4039-b60d-ab29e9169f25 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:09.992424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 704/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:12.104540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844faf0c-08db-49db-8f6d-a85ac578bdb7 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:12.105111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844faf0c-08db-49db-8f6d-a85ac578bdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:12.105111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844faf0c-08db-49db-8f6d-a85ac578bdb7 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:12.106476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 700/1404] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:12.111761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b39b29-47fc-4c5b-92c3-045fd2d371f9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:12.112567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b39b29-47fc-4c5b-92c3-045fd2d371f9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:12.113535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b39b29-47fc-4c5b-92c3-045fd2d371f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:12.195582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-18b39b29-47fc-4c5b-92c3-045fd2d371f9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:12.196823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-18b39b29-47fc-4c5b-92c3-045fd2d371f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:12.268404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:12.317435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 705/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:12.610810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844faf0c-08db-49db-8f6d-a85ac578bdb7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:12.612360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-844faf0c-08db-49db-8f6d-a85ac578bdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:12.804701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844faf0c-08db-49db-8f6d-a85ac578bdb7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:12.850309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:12.933092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:12.933937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:12.934286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:12.934566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:12.936210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:13.041376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:13.043118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e008ac-a9c0-4701-9b60-2c4855ec1be2 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:13.062440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 701/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 17:57:15.206070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc31ae5-3a55-4079-917d-cd32e7f0834a None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:15.207363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc31ae5-3a55-4079-917d-cd32e7f0834a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:15.207363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc31ae5-3a55-4079-917d-cd32e7f0834a None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:15.209082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 706/1407] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:57:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:15.214669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c3d2b-e41e-41c1-bbde-e21ba0493561 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:15.215204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c3d2b-e41e-41c1-bbde-e21ba0493561 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:15.215426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c3d2b-e41e-41c1-bbde-e21ba0493561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:15.296529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:15.304724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6a7c3d2b-e41e-41c1-bbde-e21ba0493561 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:15.310788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6a7c3d2b-e41e-41c1-bbde-e21ba0493561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:15.421341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 702/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:15.714295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc31ae5-3a55-4079-917d-cd32e7f0834a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:15.716155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8bc31ae5-3a55-4079-917d-cd32e7f0834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:15.877022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc31ae5-3a55-4079-917d-cd32e7f0834a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:15.931103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:16.000253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:16.000957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:16.001271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:16.001538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:16.003867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:16.108906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:16.110534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d4086-00d2-4578-a2f5-14d68631339d service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:16.131284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 707/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:25.419173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=44 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:26.128718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=77 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:32.321100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489d53b-28af-4bcc-a98f-27d1e2885391 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:32.321621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489d53b-28af-4bcc-a98f-27d1e2885391 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:32.321621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489d53b-28af-4bcc-a98f-27d1e2885391 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:32.322896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 703/1410] 10.222.0.15 () {54 vars in 918 bytes} [Thu Sep 1 17:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:32.327984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf43e0-43a9-44d2-b67c-5451b0e4b729 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:32.328413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf43e0-43a9-44d2-b67c-5451b0e4b729 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:32.328556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf43e0-43a9-44d2-b67c-5451b0e4b729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:32.404417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-efcf43e0-43a9-44d2-b67c-5451b0e4b729 None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:32.405620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-efcf43e0-43a9-44d2-b67c-5451b0e4b729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:32.511428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 708/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 1 17:57:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:32.526058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed68bb-f723-4bb7-883a-42134f0c1535 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:32.526339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed68bb-f723-4bb7-883a-42134f0c1535 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:32.526613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed68bb-f723-4bb7-883a-42134f0c1535 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:32.527952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 704/1412] 10.222.0.15 () {54 vars in 918 bytes} [Thu Sep 1 17:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:32.532573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5858d54-7230-4ef0-97eb-beba764b5b84 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:32.532864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5858d54-7230-4ef0-97eb-beba764b5b84 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:32.533142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5858d54-7230-4ef0-97eb-beba764b5b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:32.606181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:32.609236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c5858d54-7230-4ef0-97eb-beba764b5b84 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:32.610479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c5858d54-7230-4ef0-97eb-beba764b5b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:32.726674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 709/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 1 17:57:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:35.113505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb927237-e1c3-4dec-95e4-76602c7a3af2 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:35.114061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb927237-e1c3-4dec-95e4-76602c7a3af2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:35.114061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb927237-e1c3-4dec-95e4-76602c7a3af2 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:35.115448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:35.121063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c081-2fea-4728-b4a2-0b749e511def None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:35.121478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c081-2fea-4728-b4a2-0b749e511def None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:35.121605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c081-2fea-4728-b4a2-0b749e511def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:35.187475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f6b7c081-2fea-4728-b4a2-0b749e511def None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:35.188626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f6b7c081-2fea-4728-b4a2-0b749e511def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:35.243669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:35.287922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:35.579068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb927237-e1c3-4dec-95e4-76602c7a3af2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:35.580779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fb927237-e1c3-4dec-95e4-76602c7a3af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:35.724518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb927237-e1c3-4dec-95e4-76602c7a3af2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:35.798916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:35.838686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:35.839215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:35.839490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:35.839755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:35.841439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:35.930355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:35.931943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f386-ccd8-4b85-9596-3c2b96ebfae1 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:35.949556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 706/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:36.843935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bedec-0827-4a38-9ce6-04f7c9cba29e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:36.844519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bedec-0827-4a38-9ce6-04f7c9cba29e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:36.844519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bedec-0827-4a38-9ce6-04f7c9cba29e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:36.845892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:36.851322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e951c-58a7-497e-a63e-4df46c021e09 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:36.851737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e951c-58a7-497e-a63e-4df46c021e09 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:36.851874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e951c-58a7-497e-a63e-4df46c021e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:36.927361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a03e951c-58a7-497e-a63e-4df46c021e09 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:36.928600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a03e951c-58a7-497e-a63e-4df46c021e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:36.955066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:37.025305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:37.176079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bedec-0827-4a38-9ce6-04f7c9cba29e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:37.177632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6e6bedec-0827-4a38-9ce6-04f7c9cba29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:37.332065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bedec-0827-4a38-9ce6-04f7c9cba29e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:37.371869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:37.564411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:37.564980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:37.565307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:37.565656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:37.567251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:37.655981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:37.657588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6b10-fd94-4786-bc76-b86e5d5a19f8 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:37.676642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 712/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:38.567313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200cf5-80aa-4207-a991-e5279d85111a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:38.567934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200cf5-80aa-4207-a991-e5279d85111a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:38.567934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200cf5-80aa-4207-a991-e5279d85111a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:38.569451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:38.574994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b1111-0304-4905-9689-a067fe285983 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:38.575464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b1111-0304-4905-9689-a067fe285983 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:38.575579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b1111-0304-4905-9689-a067fe285983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:38.630084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:38.651584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ab3b1111-0304-4905-9689-a067fe285983 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:38.652816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ab3b1111-0304-4905-9689-a067fe285983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:38.748717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:38.899937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200cf5-80aa-4207-a991-e5279d85111a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:38.901802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-76200cf5-80aa-4207-a991-e5279d85111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:39.046692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200cf5-80aa-4207-a991-e5279d85111a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:39.058768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:39.159680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:39.160204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:39.160510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:39.160778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:39.162400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:39.255948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:39.258500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa72b1-9f0f-4fbb-bf60-58c4116b1dec service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:39.277713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 709/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:40.135580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99348e-9669-482a-9967-affe7ba64206 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:40.136094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99348e-9669-482a-9967-affe7ba64206 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:40.136224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99348e-9669-482a-9967-affe7ba64206 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:40.137553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:40.142641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7771f5-5a74-403f-8142-c61cc0c94c28 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:40.143166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7771f5-5a74-403f-8142-c61cc0c94c28 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:40.143286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7771f5-5a74-403f-8142-c61cc0c94c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:40.163559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:40.223514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6b7771f5-5a74-403f-8142-c61cc0c94c28 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:40.225451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6b7771f5-5a74-403f-8142-c61cc0c94c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:40.331825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:40.483247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99348e-9669-482a-9967-affe7ba64206 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:40.484790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ca99348e-9669-482a-9967-affe7ba64206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:40.602609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:40.639884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99348e-9669-482a-9967-affe7ba64206 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:40.752088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:40.752743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:40.753019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:40.753296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:40.754881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:40.829075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:40.843055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:40.844663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2b178-afac-4850-ac4c-8e9abe44810e service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:40.864079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 715/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:41.743170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98aae6-76b5-49e1-97a5-f1688cfb4100 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:41.743749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98aae6-76b5-49e1-97a5-f1688cfb4100 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:41.743749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98aae6-76b5-49e1-97a5-f1688cfb4100 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:41.745119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:41.750447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77324b09-75ef-46bc-8d29-6d436882ae95 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:41.750914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77324b09-75ef-46bc-8d29-6d436882ae95 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:41.751023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77324b09-75ef-46bc-8d29-6d436882ae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:41.827742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-77324b09-75ef-46bc-8d29-6d436882ae95 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:41.829050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-77324b09-75ef-46bc-8d29-6d436882ae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:41.923842 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:42.073455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98aae6-76b5-49e1-97a5-f1688cfb4100 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:42.074994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-aa98aae6-76b5-49e1-97a5-f1688cfb4100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:42.159167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:42.222166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98aae6-76b5-49e1-97a5-f1688cfb4100 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:42.336051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:42.336730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:42.337008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:42.337278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:42.338872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:42.387288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:42.428664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:42.430276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fce5b2-8b65-4c68-bc21-e4f9dee334e7 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:42.448360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 712/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:43.344252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc07268-1d16-4421-b5fa-82c80d9de583 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:43.344835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc07268-1d16-4421-b5fa-82c80d9de583 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:43.344835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc07268-1d16-4421-b5fa-82c80d9de583 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:43.346289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:43.351159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7684-a67c-4f0e-a6df-00cffd862258 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:43.351598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7684-a67c-4f0e-a6df-00cffd862258 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:43.351728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7684-a67c-4f0e-a6df-00cffd862258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:43.428069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-df8d7684-a67c-4f0e-a6df-00cffd862258 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:43.429436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-df8d7684-a67c-4f0e-a6df-00cffd862258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:43.533675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:43.691383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc07268-1d16-4421-b5fa-82c80d9de583 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:43.692930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0bc07268-1d16-4421-b5fa-82c80d9de583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:43.741138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:43.836294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc07268-1d16-4421-b5fa-82c80d9de583 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:43.951045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:43.951651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:43.951932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:43.952196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:43.953808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:43.961297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:44.045259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:44.046860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8bf5f-7c5b-4fb0-b9e4-0ac9a604aca2 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:44.065320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 718/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:44.915514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584bdd-120a-4dc4-b337-31314b512249 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:44.916051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584bdd-120a-4dc4-b337-31314b512249 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:44.916051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9584bdd-120a-4dc4-b337-31314b512249 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:44.917451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:44.922353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f99ea7-3659-4413-95ae-131a9ac5213c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:44.922753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f99ea7-3659-4413-95ae-131a9ac5213c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:44.922930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f99ea7-3659-4413-95ae-131a9ac5213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:44.999267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-38f99ea7-3659-4413-95ae-131a9ac5213c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:45.000501 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-38f99ea7-3659-4413-95ae-131a9ac5213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:45.069727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:45.098398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:57:45.247485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9584bdd-120a-4dc4-b337-31314b512249 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:45.248989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e9584bdd-120a-4dc4-b337-31314b512249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:45.266540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:45.402003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9584bdd-120a-4dc4-b337-31314b512249 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:45.489523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:45.520007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:45.520678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:45.520959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:45.521231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:45.522925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:45.612886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:45.614511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34692ae0-8aa5-4048-9c41-9cb35a9ed9d8 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:45.632859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 715/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:46.510449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625e35b-93d2-4e6b-ace7-e816b2e5fccd None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:46.511035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625e35b-93d2-4e6b-ace7-e816b2e5fccd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:46.511035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625e35b-93d2-4e6b-ace7-e816b2e5fccd None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:46.512358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:46.517372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f65ee8-561b-4d60-bde7-7d107a23e457 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:46.517820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f65ee8-561b-4d60-bde7-7d107a23e457 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:46.518592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f65ee8-561b-4d60-bde7-7d107a23e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:46.595094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d3f65ee8-561b-4d60-bde7-7d107a23e457 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:46.596607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d3f65ee8-561b-4d60-bde7-7d107a23e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:46.638072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:46.694200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:46.843706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625e35b-93d2-4e6b-ace7-e816b2e5fccd None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:46.845166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c625e35b-93d2-4e6b-ace7-e816b2e5fccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:46.991176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625e35b-93d2-4e6b-ace7-e816b2e5fccd None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:47.048514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:47.107467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:47.108001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:47.108277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:47.108585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:47.110179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:47.199644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:47.202324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d2360-cdb5-447b-8d0e-0503ec5eb3ae service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:47.219953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 721/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:48.074962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa166a-cfe5-40a3-b864-776c21e677ad None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:48.074962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa166a-cfe5-40a3-b864-776c21e677ad None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:48.074962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa166a-cfe5-40a3-b864-776c21e677ad None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:48.076325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 17:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:48.081379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a01f1-65f5-44ab-afc9-bafc2726016e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:48.081802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a01f1-65f5-44ab-afc9-bafc2726016e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:48.081933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a01f1-65f5-44ab-afc9-bafc2726016e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:48.156925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e03a01f1-65f5-44ab-afc9-bafc2726016e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:48.158150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e03a01f1-65f5-44ab-afc9-bafc2726016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:48.160306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:48.258882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 17:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:48.409178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa166a-cfe5-40a3-b864-776c21e677ad None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:48.410696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1afa166a-cfe5-40a3-b864-776c21e677ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:48.557955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa166a-cfe5-40a3-b864-776c21e677ad None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:48.585752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:48.685442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:48.685966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:48.686247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:48.686519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:48.688128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:48.780015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:48.781618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58f2c2-3684-4f9e-962c-c3b77fc1c897 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:48.799839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 718/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:50.627506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b487-4ae1-47f1-9830-906936a24c21 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:50.628137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b487-4ae1-47f1-9830-906936a24c21 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:50.628137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b487-4ae1-47f1-9830-906936a24c21 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:50.629437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:50.634832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd253b72-2bed-49fa-855f-bd1da03da14b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:50.635516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd253b72-2bed-49fa-855f-bd1da03da14b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:50.635516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd253b72-2bed-49fa-855f-bd1da03da14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:50.678250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:50.704692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-dd253b72-2bed-49fa-855f-bd1da03da14b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:50.706530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dd253b72-2bed-49fa-855f-bd1da03da14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:50.803357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:51.098234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b487-4ae1-47f1-9830-906936a24c21 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:51.099772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8546b487-4ae1-47f1-9830-906936a24c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:51.205898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b487-4ae1-47f1-9830-906936a24c21 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:51.239397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:51.284246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:51.284886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:51.285163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:51.285432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:51.287012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:51.398693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:51.400770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a95b1-04ff-4a92-850a-ead18d23d2cd service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:51.427011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 724/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:53.669716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf21e2-fb65-4761-b339-88da8de65836 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:53.670285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf21e2-fb65-4761-b339-88da8de65836 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:53.670285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf21e2-fb65-4761-b339-88da8de65836 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:53.680961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:57:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:53.687009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc126371-2eef-4f79-96ce-dce9e81bcd42 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:53.687470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc126371-2eef-4f79-96ce-dce9e81bcd42 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:53.687595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc126371-2eef-4f79-96ce-dce9e81bcd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:53.759254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bc126371-2eef-4f79-96ce-dce9e81bcd42 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:53.760522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bc126371-2eef-4f79-96ce-dce9e81bcd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:53.774443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:53.858666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:54.143375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf21e2-fb65-4761-b339-88da8de65836 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:54.144989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-92bf21e2-fb65-4761-b339-88da8de65836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:54.248301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf21e2-fb65-4761-b339-88da8de65836 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:54.261259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:54.323028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:54.323631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:54.323905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:54.324173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:54.325787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:54.444136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:54.445624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbac8e-4cf2-42d2-8193-c3c260c8ff96 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:54.463594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 721/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:56.736071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb658-7fbd-4942-80e3-56dd6e1ce3b1 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:56.737075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb658-7fbd-4942-80e3-56dd6e1ce3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:56.737075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb658-7fbd-4942-80e3-56dd6e1ce3b1 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:56.738162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:56.743391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fcef1-e546-47e9-8cfa-fb273a7cb038 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:56.743797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fcef1-e546-47e9-8cfa-fb273a7cb038 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:56.743926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fcef1-e546-47e9-8cfa-fb273a7cb038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:56.792143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:56.809486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-915fcef1-e546-47e9-8cfa-fb273a7cb038 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:56.810682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-915fcef1-e546-47e9-8cfa-fb273a7cb038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:56.907330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:57.196054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb658-7fbd-4942-80e3-56dd6e1ce3b1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:57.197647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a49eb658-7fbd-4942-80e3-56dd6e1ce3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:57.303796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb658-7fbd-4942-80e3-56dd6e1ce3b1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:57:57.342866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:57.376683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:57:57.377532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:57.377813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:57.378085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:57.379679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:57.490346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:57:57.491662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d88e3-62bd-4a55-b9ca-7b048a15a194 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:57:57.509580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 727/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:57:59.538725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730818b8-f751-4514-acf7-d8d99e8e074d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:59.539263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730818b8-f751-4514-acf7-d8d99e8e074d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:59.539263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730818b8-f751-4514-acf7-d8d99e8e074d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:59.540608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:57:59.545669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe9ca-4f5c-4dfd-97d1-a3d05915aca5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:57:59.546109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe9ca-4f5c-4dfd-97d1-a3d05915aca5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:57:59.546208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe9ca-4f5c-4dfd-97d1-a3d05915aca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:57:59.610802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-eb8fe9ca-4f5c-4dfd-97d1-a3d05915aca5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:57:59.612138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-eb8fe9ca-4f5c-4dfd-97d1-a3d05915aca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:57:59.632336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:57:59.708028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:57:59.990911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730818b8-f751-4514-acf7-d8d99e8e074d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:57:59.992453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-730818b8-f751-4514-acf7-d8d99e8e074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:00.093363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730818b8-f751-4514-acf7-d8d99e8e074d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:00.113542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:00.167689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:00.168293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:00.168600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:00.168870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:00.170450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:00.281617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:00.283246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a1f46-299b-4453-905b-0601eb23eacb service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:00.301765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 724/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:02.277625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f712b7-c0dd-4894-b63a-7ebb4bafddad None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:02.278080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f712b7-c0dd-4894-b63a-7ebb4bafddad None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:02.278187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f712b7-c0dd-4894-b63a-7ebb4bafddad None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:02.279504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:02.284906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db1bdc-9f5f-4863-8de1-8b79afc9412e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:02.285343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db1bdc-9f5f-4863-8de1-8b79afc9412e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:02.285463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db1bdc-9f5f-4863-8de1-8b79afc9412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:02.349949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:02.352531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-16db1bdc-9f5f-4863-8de1-8b79afc9412e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:02.353736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-16db1bdc-9f5f-4863-8de1-8b79afc9412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:02.451520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:02.733382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f712b7-c0dd-4894-b63a-7ebb4bafddad None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:02.734946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e7f712b7-c0dd-4894-b63a-7ebb4bafddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:02.836688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f712b7-c0dd-4894-b63a-7ebb4bafddad None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:02.884668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:02.908778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:02.909380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:02.909660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:02.909931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:02.911547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:03.019472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:03.020834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0db17e-b1b9-4728-aa79-e694243eab4f service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:03.039471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 730/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:05.009307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc1ddb-efde-445d-9a73-b81209062db2 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:05.009875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc1ddb-efde-445d-9a73-b81209062db2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:05.009875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc1ddb-efde-445d-9a73-b81209062db2 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:05.011197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:05.016284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1c35-a9d2-4e7e-81c3-0b0857218749 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:05.016681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1c35-a9d2-4e7e-81c3-0b0857218749 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:05.016857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1c35-a9d2-4e7e-81c3-0b0857218749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:05.081892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-12ce1c35-a9d2-4e7e-81c3-0b0857218749 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:05.083081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-12ce1c35-a9d2-4e7e-81c3-0b0857218749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:05.112672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:05.178863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:05.462537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc1ddb-efde-445d-9a73-b81209062db2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:05.464134 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e7fc1ddb-efde-445d-9a73-b81209062db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:05.574609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc1ddb-efde-445d-9a73-b81209062db2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:05.602988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:05.648605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:05.649208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:05.649488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:05.650027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:05.651630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:05.759675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:05.761134 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5036c-544d-438f-a312-54b0b6cdf2b5 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:05.779965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 727/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:07.798116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02a18-4e54-47cf-b0b8-92c46493df3e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:07.798928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02a18-4e54-47cf-b0b8-92c46493df3e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:07.798928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02a18-4e54-47cf-b0b8-92c46493df3e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:07.800035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:07.806312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfccc4-e8b5-431c-97d5-aa018b007cdb None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:07.806804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfccc4-e8b5-431c-97d5-aa018b007cdb None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:07.807220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfccc4-e8b5-431c-97d5-aa018b007cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:07.886527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7ebfccc4-e8b5-431c-97d5-aa018b007cdb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:07.887745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7ebfccc4-e8b5-431c-97d5-aa018b007cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:07.898141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:07.999914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:08.304671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02a18-4e54-47cf-b0b8-92c46493df3e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:08.306417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3de02a18-4e54-47cf-b0b8-92c46493df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:08.419970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02a18-4e54-47cf-b0b8-92c46493df3e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:08.477336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:08.495742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:08.496386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:08.496669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:08.496944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:08.498637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:08.607695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:08.609215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164b276-449a-4ea1-a2b6-661ca9287013 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:08.629188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 733/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:10.635999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cbb16-52d6-463d-9dee-1f20e9d7d57d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:10.636578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cbb16-52d6-463d-9dee-1f20e9d7d57d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:10.636578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cbb16-52d6-463d-9dee-1f20e9d7d57d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:10.637924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:10.643586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50835-744e-4a46-9fa8-f0bec4c5097a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:10.644018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50835-744e-4a46-9fa8-f0bec4c5097a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:10.644127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50835-744e-4a46-9fa8-f0bec4c5097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:10.710678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-66d50835-744e-4a46-9fa8-f0bec4c5097a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:10.711891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-66d50835-744e-4a46-9fa8-f0bec4c5097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:10.751288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:10.809066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:58:11.092112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cbb16-52d6-463d-9dee-1f20e9d7d57d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:11.093738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4b8cbb16-52d6-463d-9dee-1f20e9d7d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:11.204050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cbb16-52d6-463d-9dee-1f20e9d7d57d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:11.245183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:11.282313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:11.283797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:11.284105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:11.284411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:11.286016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:11.393477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:11.394910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe894f5-5596-4eef-980c-977503e5c931 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:11.413932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 730/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:13.365523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412f8e2-69cc-4ac3-b24b-1a699686eabe None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:13.366025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412f8e2-69cc-4ac3-b24b-1a699686eabe None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:13.366025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412f8e2-69cc-4ac3-b24b-1a699686eabe None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:13.367375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:13.372806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980a86c-888d-4a10-ae30-a9177ea50f2c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:13.373200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980a86c-888d-4a10-ae30-a9177ea50f2c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:13.373385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980a86c-888d-4a10-ae30-a9177ea50f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:13.439771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d980a86c-888d-4a10-ae30-a9177ea50f2c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:13.441005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d980a86c-888d-4a10-ae30-a9177ea50f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:13.458811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:13.538019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:13.839052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412f8e2-69cc-4ac3-b24b-1a699686eabe None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:13.840645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d412f8e2-69cc-4ac3-b24b-1a699686eabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:13.945814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412f8e2-69cc-4ac3-b24b-1a699686eabe None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:14.011940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:14.018753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:14.019352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:14.024791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:14.024791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:14.024791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:14.129568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:14.130991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce780832-1516-40f4-854f-384fb2c2d1e6 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:14.150047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 736/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:16.073706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93088206-b80a-4d6f-ba5c-d9152a4039a9 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:16.074365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93088206-b80a-4d6f-ba5c-d9152a4039a9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:16.074365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93088206-b80a-4d6f-ba5c-d9152a4039a9 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:16.075577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 17:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:16.081178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1662d-e493-48c6-b91d-0db326acad00 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:16.081591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1662d-e493-48c6-b91d-0db326acad00 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:16.081762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1662d-e493-48c6-b91d-0db326acad00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:16.150183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-03e1662d-e493-48c6-b91d-0db326acad00 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:16.151368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-03e1662d-e493-48c6-b91d-0db326acad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:16.200067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:16.248687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:16.531039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93088206-b80a-4d6f-ba5c-d9152a4039a9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:16.532800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-93088206-b80a-4d6f-ba5c-d9152a4039a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:16.633765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93088206-b80a-4d6f-ba5c-d9152a4039a9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:16.681739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:16.707921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:16.708552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:16.708828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:16.709093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:16.711303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:16.822729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:16.824083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c59c08-83af-45c2-bce4-7ba964a51a3a service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:16.843080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 733/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:26.246835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=22 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:26.840806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=66 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:36.626396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce3ea6-293d-44fe-a7ef-bbd7c319860f None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:36.626970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce3ea6-293d-44fe-a7ef-bbd7c319860f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:36.626970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce3ea6-293d-44fe-a7ef-bbd7c319860f None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:36.628277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 738/1471] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:36.633386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f12e5-bddd-443a-b1ce-1e41278009c4 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:36.633792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f12e5-bddd-443a-b1ce-1e41278009c4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:36.633909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f12e5-bddd-443a-b1ce-1e41278009c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:36.700201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-fb6f12e5-bddd-443a-b1ce-1e41278009c4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:36.701417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fb6f12e5-bddd-443a-b1ce-1e41278009c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:36.798430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 734/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:37.098967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b16afb-7c15-4ccf-8daa-4930755596bd None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:37.099248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b16afb-7c15-4ccf-8daa-4930755596bd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:37.099521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b16afb-7c15-4ccf-8daa-4930755596bd None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:37.100909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 739/1473] 10.222.0.15 () {54 vars in 967 bytes} [Thu Sep 1 17:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:37.105742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd9023-f01b-468b-8d80-4977d5b2f0c4 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:37.106026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd9023-f01b-468b-8d80-4977d5b2f0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:37.106299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd9023-f01b-468b-8d80-4977d5b2f0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:37.183726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a7fd9023-f01b-468b-8d80-4977d5b2f0c4 None None] MFA Rules not processed for user `dbb0ec082bdb45b0bb2c59cbad440d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:37.184978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a7fd9023-f01b-468b-8d80-4977d5b2f0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:37.206514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:37.264226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 735/1474] 10.222.0.15 () {58 vars in 1064 bytes} [Thu Sep 1 17:58:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:37.271753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b16afb-7c15-4ccf-8daa-4930755596bd None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:37.273337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-71b16afb-7c15-4ccf-8daa-4930755596bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:37.377739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b16afb-7c15-4ccf-8daa-4930755596bd None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:37.450845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:37.451447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:37.451727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:37.451999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:37.454147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:37.508405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:37.564228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:37.565661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eaeaac-862d-4d23-bb35-2d6520116d24 service glance] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:37.584277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 740/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:39.565276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19339e1-38a1-4898-a181-7ec98d01b653 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:39.565804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19339e1-38a1-4898-a181-7ec98d01b653 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:39.565804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19339e1-38a1-4898-a181-7ec98d01b653 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:39.567086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 736/1476] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:39.572426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09c7e-1295-4d78-9899-cb01f78e160f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:39.572856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09c7e-1295-4d78-9899-cb01f78e160f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:39.572968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09c7e-1295-4d78-9899-cb01f78e160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:39.637848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f7f09c7e-1295-4d78-9899-cb01f78e160f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:39.639078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f7f09c7e-1295-4d78-9899-cb01f78e160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:39.742507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 741/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 17:58:40.015566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19339e1-38a1-4898-a181-7ec98d01b653 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:40.018018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c19339e1-38a1-4898-a181-7ec98d01b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:40.151705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19339e1-38a1-4898-a181-7ec98d01b653 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:40.199841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:40.243678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:40.244282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:40.244595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:40.244860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:40.245493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:40.247264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 17:58:40.247264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 17:58:40.247264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 17:58:40.248406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdab7d-3307-403e-ae7e-8570cf74227c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:40.269965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 737/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 1 17:58:40 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 17:58:42.202253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b00643-d979-45ca-868e-a51663392de9 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:42.202815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b00643-d979-45ca-868e-a51663392de9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:42.202815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b00643-d979-45ca-868e-a51663392de9 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:42.204159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 742/1479] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:42.209265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f00c7-e215-4744-9be5-df274ea24e8e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:42.209663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f00c7-e215-4744-9be5-df274ea24e8e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:42.209823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f00c7-e215-4744-9be5-df274ea24e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:42.278222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e08f00c7-e215-4744-9be5-df274ea24e8e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:42.279437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e08f00c7-e215-4744-9be5-df274ea24e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:42.386817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 738/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:42.691473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b00643-d979-45ca-868e-a51663392de9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:42.693023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-45b00643-d979-45ca-868e-a51663392de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:42.704698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:42.843887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b00643-d979-45ca-868e-a51663392de9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:42.924294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:42.956751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:42.957593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:42.958180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:42.958606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:42.960800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:43.051062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:43.052653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b773c1-5b6d-4b6a-9a73-55165fb25c49 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:43.070588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 743/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:45.156594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bac7d7-84fd-4fc6-b6bf-68af2a2df615 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:45.157141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bac7d7-84fd-4fc6-b6bf-68af2a2df615 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:45.157271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bac7d7-84fd-4fc6-b6bf-68af2a2df615 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:45.158568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 739/1482] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:45.163802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83276428-6931-43ce-bb4c-87c39d6aa870 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:45.164225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83276428-6931-43ce-bb4c-87c39d6aa870 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:45.164380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83276428-6931-43ce-bb4c-87c39d6aa870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:45.230882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-83276428-6931-43ce-bb4c-87c39d6aa870 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:45.232086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-83276428-6931-43ce-bb4c-87c39d6aa870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:45.279068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:45.329741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 744/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:45.637684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bac7d7-84fd-4fc6-b6bf-68af2a2df615 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:45.639284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c1bac7d7-84fd-4fc6-b6bf-68af2a2df615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:45.663915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:45.751393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bac7d7-84fd-4fc6-b6bf-68af2a2df615 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:45.824137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:45.824754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:45.825030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:45.825304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:45.826911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:45.902627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:45.937739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:45.939527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8b561-b720-4cc3-8512-11c9ffb8c4a6 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:45.959097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 740/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:47.910600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f92277-1b6c-4879-9ffd-ab7da06eb4d7 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:47.911238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f92277-1b6c-4879-9ffd-ab7da06eb4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:47.911238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f92277-1b6c-4879-9ffd-ab7da06eb4d7 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:47.912512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 745/1485] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:47.917709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d748-5aa5-4716-a06b-b8cb25c75449 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:47.918127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d748-5aa5-4716-a06b-b8cb25c75449 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:47.918240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d748-5aa5-4716-a06b-b8cb25c75449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:47.985193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2f31d748-5aa5-4716-a06b-b8cb25c75449 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:47.986390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2f31d748-5aa5-4716-a06b-b8cb25c75449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:48.213065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 741/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:48.505020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f92277-1b6c-4879-9ffd-ab7da06eb4d7 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:48.506543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-38f92277-1b6c-4879-9ffd-ab7da06eb4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:48.610088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f92277-1b6c-4879-9ffd-ab7da06eb4d7 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:48.680477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:48.682213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:48.682813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:48.683087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:48.683355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:48.684973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:48.790765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:48.792089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a2a7-9cc4-4089-b17c-a265acb9c971 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:48.810004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 746/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:50.838454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30456087-3f9f-43ef-9e63-4ac3bea56c3a None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:50.839028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30456087-3f9f-43ef-9e63-4ac3bea56c3a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:50.839127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30456087-3f9f-43ef-9e63-4ac3bea56c3a None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:50.840482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 742/1488] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:50.846119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227445-6f55-4df5-b64c-050ef3949854 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:50.846581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227445-6f55-4df5-b64c-050ef3949854 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:50.846686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227445-6f55-4df5-b64c-050ef3949854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:50.912506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-70227445-6f55-4df5-b64c-050ef3949854 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:50.913703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-70227445-6f55-4df5-b64c-050ef3949854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:50.965854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:51.010606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 747/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:51.314573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30456087-3f9f-43ef-9e63-4ac3bea56c3a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:51.316116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-30456087-3f9f-43ef-9e63-4ac3bea56c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:51.352058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:51.420305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30456087-3f9f-43ef-9e63-4ac3bea56c3a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:51.492800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:51.493402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:51.493680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:51.493950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:51.495532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:51.575326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:51.604250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:51.605734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228e7fe-325b-4faa-83ac-5857192f7dfa service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:51.624201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 743/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:58:53.921966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3e7a-c51e-45df-ac6e-46c4fa510c06 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:53.922666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3e7a-c51e-45df-ac6e-46c4fa510c06 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:53.922666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3e7a-c51e-45df-ac6e-46c4fa510c06 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:53.923929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 748/1491] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 1 17:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 17:58:53.929055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5168a3-87f6-45c5-bef5-de7815e91534 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:53.929451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5168a3-87f6-45c5-bef5-de7815e91534 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:53.929602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5168a3-87f6-45c5-bef5-de7815e91534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:54.002323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ba5168a3-87f6-45c5-bef5-de7815e91534 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 17:58:54.003522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ba5168a3-87f6-45c5-bef5-de7815e91534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:54.100949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 744/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 1 17:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 17:58:54.393764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3e7a-c51e-45df-ac6e-46c4fa510c06 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:58:54.395329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7a1e3e7a-c51e-45df-ac6e-46c4fa510c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:54.499712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3e7a-c51e-45df-ac6e-46c4fa510c06 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:58:54.573199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:58:54.573731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:58:54.574013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:58:54.574286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:58:54.575857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:58:54.584844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:58:54.683496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:58:54.684960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdfb0c-7377-47e5-b125-a7c69a225104 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:58:54.702686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 749/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:59:04.099020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=82 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:59:04.700376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=50 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:17.168728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:17.169333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:17.169333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:17.245702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:00:17.246986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:17.345090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 745/1494] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:00:17.353509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:17.354006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:17.354103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:17.429459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:00:17.430613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:17.526407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 750/1495] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:00:17.531540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:17.532793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:17.544149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae34a6-8419-4f02-8287-5d6b1aea75b4 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:17.605157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:17.639053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:17.639651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:17.639928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:17.640197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:17.640985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:00:17.641473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:17.642768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:17.642768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:00:17.642768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:00:17.643657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:17.655789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 746/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:00:17.663981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:17.674120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221862f-6f23-4b6f-b0bc-112f353d5d3a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:17.735064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:17.767549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:17.768147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:17.768450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:17.768722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:17.769340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:17.771029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:17.771029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:00:17.771029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:00:17.772215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:17.969626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 751/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:00:17 2022] POST /identity/v3/projects => generated 377 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:00:17.973910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:17.975491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:18.009770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae046bc-fd62-4969-973c-c320cfd1cc31 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:18.106434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:18.107662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.107944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.108214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:18.108889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:18.109897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:18.109897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:00:18.109897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:00:18.111129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:18.243865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 747/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:00:17 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:00:18.248143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:18.249764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:18.312269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5fa3-aa34-4dc9-a071-708d83898b7f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:18.396725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:18.408650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:18.409260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.409529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.409792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:18.410429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:18.411450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:18.411450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:00:18.411450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:00:18.412343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:18.425296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 752/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:00:18.429390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:18.430917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:18.443161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3a8ef-201c-4224-b92e-6ef94135437a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:18.468830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:18.540080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:18.540766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.541044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.541312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:18.541962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:18.542968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:18.542968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:00:18.542968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:00:18.543848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5edbf-7ba2-4be6-856f-5fa200dff05f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:18.557044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 748/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:00:18.561929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:18.563493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:18.597363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07acb3-b8d6-4478-ab8e-d99bc05312b6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:18.697471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:18.698141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.698422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.698698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] PATH_INFO: `/v3/projects/95b2026d44c94ddfa020605ee6b1d0d0/users/c03ca46d568f48c6ba60ede750deab7a/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:18.723834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b2026d44c94ddfa020605ee6b1d0d0, user_id=c03ca46d568f48c6ba60ede750deab7a, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:18.724882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:18.724882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:00:18.724882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:00:18.726919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1076abe-3b67-48e6-8f22-4597754d01ac admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:18.818754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 753/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:00:18 2022] PUT /identity/v3/projects/95b2026d44c94ddfa020605ee6b1d0d0/users/c03ca46d568f48c6ba60ede750deab7a/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:00:18.831166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627984f0-da85-4591-868d-e9ebd79ace3d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.831450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627984f0-da85-4591-868d-e9ebd79ace3d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.831720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627984f0-da85-4591-868d-e9ebd79ace3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:18.887692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-627984f0-da85-4591-868d-e9ebd79ace3d None None] MFA Rules not processed for user `c03ca46d568f48c6ba60ede750deab7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:00:18.889090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-627984f0-da85-4591-868d-e9ebd79ace3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:18.965276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 749/1502] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:00:18.979113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4583-8fa9-4484-8ca7-cf2162869f84 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.979404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4583-8fa9-4484-8ca7-cf2162869f84 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.979681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4583-8fa9-4484-8ca7-cf2162869f84 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:18.981054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 754/1503] 10.222.0.15 () {54 vars in 964 bytes} [Thu Sep 1 18:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:00:18.986614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38808-23c7-4439-9d12-8fb55db3c45d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:18.986901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38808-23c7-4439-9d12-8fb55db3c45d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:18.987178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38808-23c7-4439-9d12-8fb55db3c45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:19.018679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:19.062719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-cfe38808-23c7-4439-9d12-8fb55db3c45d None None] MFA Rules not processed for user `9c32115f24264e738c432f700a2f7dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:00:19.063921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cfe38808-23c7-4439-9d12-8fb55db3c45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:19.139365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 750/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Thu Sep 1 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:00:19.148864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e4583-8fa9-4484-8ca7-cf2162869f84 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:19.150374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e09e4583-8fa9-4484-8ca7-cf2162869f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:19.200233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:19.254915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e4583-8fa9-4484-8ca7-cf2162869f84 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:19.328241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:19.328875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:19.329153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:19.329425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:19.331000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:19.431669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:19.433143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:19.436941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:19.451429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 755/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:00:19.499571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a7ec1-e163-43fb-9f47-3afcd61e41f3 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:19.500011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a7ec1-e163-43fb-9f47-3afcd61e41f3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:19.500112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a7ec1-e163-43fb-9f47-3afcd61e41f3 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:19.501411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 751/1506] 10.222.0.15 () {52 vars in 830 bytes} [Thu Sep 1 18:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:00:19.504941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:19.506183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:19.603392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddf9a5-8e56-4f1b-b325-9382068d6d7a service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:19.695937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:19.696514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:19.696792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:19.697060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] PATH_INFO: `/v3/users/c03ca46d568f48c6ba60ede750deab7a` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:19.709033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03ca46d568f48c6ba60ede750deab7a)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:19.710044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:19.710044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:00:19.710044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:00:19.711241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:19.721351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:19.964617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03ca46d568f48c6ba60ede750deab7a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:19.978860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03ca46d568f48c6ba60ede750deab7a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:19.986016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03ca46d568f48c6ba60ede750deab7a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:19.986016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03ca46d568f48c6ba60ede750deab7a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:20.076000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03ca46d568f48c6ba60ede750deab7a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:20.105179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 756/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:00:19 2022] DELETE /identity/v3/users/c03ca46d568f48c6ba60ede750deab7a => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:00:20.110449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:20.110735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:20.111010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:20.170588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:00:20.171785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:20.249018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:20.262914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 752/1508] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:00:20.273199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:20.274423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:20.375293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9900-8548-48e1-bd75-937a24c585ce admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:20.430661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:20.449938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:20.450553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:20.451382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:20.451874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:20.453529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:20.563843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:20.565246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcada3-8c16-4034-98c0-d073620a3c94 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:20.584238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 757/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:00:21.084266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:00:21.091904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:00:21.097313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22036f83-73c6-479f-9f3d-0e638107c6f7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:00:21.197005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:00:21.197635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:00:21.198015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:00:21.198288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] PATH_INFO: `/v3/projects/95b2026d44c94ddfa020605ee6b1d0d0` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:00:21.208784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b2026d44c94ddfa020605ee6b1d0d0)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:00:21.209821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:00:21.209821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:00:21.209821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:00:21.211021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:00:21.372316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b2026d44c94ddfa020605ee6b1d0d0', 'request_id': 'req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:21.452903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b2026d44c94ddfa020605ee6b1d0d0', 'request_id': 'req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:21.459237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b2026d44c94ddfa020605ee6b1d0d0', 'request_id': 'req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:21.488677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] The token cache is being invalidate because project 95b2026d44c94ddfa020605ee6b1d0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:00:21.488962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:21.489385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-356cd6c2-4098-49e7-98ff-b6ce0ebc84bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:00:21.490789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 753/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:00:21 2022] DELETE /identity/v3/projects/95b2026d44c94ddfa020605ee6b1d0d0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:00:30.582121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=62 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:00:31.486674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:01:52.041849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd639a90-731b-456f-91e5-2bdda3efdcd1 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:52.042407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd639a90-731b-456f-91e5-2bdda3efdcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:52.042407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd639a90-731b-456f-91e5-2bdda3efdcd1 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:52.043699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 18:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:01:52.048575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1eadd-b0c2-4dfb-a44c-2bab926cadee None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:52.049016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1eadd-b0c2-4dfb-a44c-2bab926cadee None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:52.049121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1eadd-b0c2-4dfb-a44c-2bab926cadee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:52.127747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c4d1eadd-b0c2-4dfb-a44c-2bab926cadee None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:01:52.129496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c4d1eadd-b0c2-4dfb-a44c-2bab926cadee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:01:52.236622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:01:52.393067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd639a90-731b-456f-91e5-2bdda3efdcd1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:01:52.395421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cd639a90-731b-456f-91e5-2bdda3efdcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:01:52.528930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd639a90-731b-456f-91e5-2bdda3efdcd1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:01:52.534719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a30ff-baf0-4fb9-90d8-c540be41f04f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:01:52.536204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a30ff-baf0-4fb9-90d8-c540be41f04f service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:52.536512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a30ff-baf0-4fb9-90d8-c540be41f04f service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:52.536782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a30ff-baf0-4fb9-90d8-c540be41f04f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:52.545451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a30ff-baf0-4fb9-90d8-c540be41f04f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:01:52.547243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a30ff-baf0-4fb9-90d8-c540be41f04f service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:01:52.557881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 759/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:01:53.413122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa843280-a301-4cc5-834c-66e795e91323 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:53.413695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa843280-a301-4cc5-834c-66e795e91323 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:53.413695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa843280-a301-4cc5-834c-66e795e91323 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:53.415023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 18:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:01:53.419898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cb031-f061-4096-8710-bd35faac8c64 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:53.420297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cb031-f061-4096-8710-bd35faac8c64 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:53.420479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cb031-f061-4096-8710-bd35faac8c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:53.440737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f99cb031-f061-4096-8710-bd35faac8c64 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:01:53.442462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f99cb031-f061-4096-8710-bd35faac8c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:01:53.450811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:01:53.601041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa843280-a301-4cc5-834c-66e795e91323 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:01:53.616079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa843280-a301-4cc5-834c-66e795e91323 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:01:53.621416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af2e49-3201-4921-8f1f-e50e9797fddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:01:53.622040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2e49-3201-4921-8f1f-e50e9797fddf service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:53.622320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2e49-3201-4921-8f1f-e50e9797fddf service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:53.622594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2e49-3201-4921-8f1f-e50e9797fddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:53.631033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af2e49-3201-4921-8f1f-e50e9797fddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:01:53.632696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af2e49-3201-4921-8f1f-e50e9797fddf service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:01:53.642208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 756/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:01:55.243783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6697bf-78a1-4141-b799-2ea20ae6a888 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:55.244363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6697bf-78a1-4141-b799-2ea20ae6a888 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:55.244363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6697bf-78a1-4141-b799-2ea20ae6a888 None None] PATH_INFO: `/v3` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:55.245616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 761/1517] 10.222.0.74 () {54 vars in 925 bytes} [Thu Sep 1 18:01:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:01:55.251753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc3b1-62ae-4b9c-b780-7a9b656e2b16 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:55.252165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc3b1-62ae-4b9c-b780-7a9b656e2b16 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:55.252331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc3b1-62ae-4b9c-b780-7a9b656e2b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:55.311862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9fcbc3b1-62ae-4b9c-b780-7a9b656e2b16 None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:01:55.313111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9fcbc3b1-62ae-4b9c-b780-7a9b656e2b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:01:55.330151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:01:55.384686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 757/1518] 10.222.0.74 () {58 vars in 1015 bytes} [Thu Sep 1 18:01:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:01:59.606951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6172-2936-423f-8dfa-f4c0ad229bad None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:59.607472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6172-2936-423f-8dfa-f4c0ad229bad None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:59.607472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf6172-2936-423f-8dfa-f4c0ad229bad None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:59.608804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 762/1519] 10.222.0.15 () {54 vars in 961 bytes} [Thu Sep 1 18:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:01:59.614463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881b7f6-ac7c-47d6-a33c-8eeebc08b1be None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:59.614900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881b7f6-ac7c-47d6-a33c-8eeebc08b1be None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:59.614998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881b7f6-ac7c-47d6-a33c-8eeebc08b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:59.634306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c881b7f6-ac7c-47d6-a33c-8eeebc08b1be None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:01:59.635491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c881b7f6-ac7c-47d6-a33c-8eeebc08b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:01:59.644121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 758/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Thu Sep 1 18:01:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:01:59.651596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf6172-2936-423f-8dfa-f4c0ad229bad None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:01:59.665536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf6172-2936-423f-8dfa-f4c0ad229bad None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:01:59.670470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d37870-0f4d-4c51-9876-2ecc7b057f6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:01:59.671055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d37870-0f4d-4c51-9876-2ecc7b057f6e service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:01:59.671335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d37870-0f4d-4c51-9876-2ecc7b057f6e service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:01:59.671613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d37870-0f4d-4c51-9876-2ecc7b057f6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:01:59.680321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d37870-0f4d-4c51-9876-2ecc7b057f6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:01:59.681885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d37870-0f4d-4c51-9876-2ecc7b057f6e service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:01:59.691342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 763/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:01:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:04.877696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ce50-2ff5-4e5d-80b0-05a81301a86b None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:04.878263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ce50-2ff5-4e5d-80b0-05a81301a86b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:04.878263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ce50-2ff5-4e5d-80b0-05a81301a86b None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:04.880112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 18:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:02:04.884979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b734b4-29bc-41b9-b625-68cdffe43f3c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:04.885357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b734b4-29bc-41b9-b625-68cdffe43f3c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:04.885567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b734b4-29bc-41b9-b625-68cdffe43f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:04.905511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-15b734b4-29bc-41b9-b625-68cdffe43f3c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:02:04.906751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-15b734b4-29bc-41b9-b625-68cdffe43f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:04.914717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 18:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:02:05.070556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242ce50-2ff5-4e5d-80b0-05a81301a86b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:02:05.085635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242ce50-2ff5-4e5d-80b0-05a81301a86b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:02:05.092430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cdc68-6940-4f1b-aeb4-ab9209056f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:02:05.093567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cdc68-6940-4f1b-aeb4-ab9209056f83 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:05.093856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cdc68-6940-4f1b-aeb4-ab9209056f83 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:05.094135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cdc68-6940-4f1b-aeb4-ab9209056f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:05.106184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cdc68-6940-4f1b-aeb4-ab9209056f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:02:05.107770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cdc68-6940-4f1b-aeb4-ab9209056f83 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:02:05.117662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 760/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:05.939389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c155cf6-4ae0-4e7a-bcbe-bef13f962ff2 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:05.939956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c155cf6-4ae0-4e7a-bcbe-bef13f962ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:05.939956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c155cf6-4ae0-4e7a-bcbe-bef13f962ff2 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:05.941462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 18:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:02:05.946380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1260e-48f2-4cfe-a7d0-3b17e038fbb5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:05.946806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1260e-48f2-4cfe-a7d0-3b17e038fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:05.946936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1260e-48f2-4cfe-a7d0-3b17e038fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:05.966600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3ec1260e-48f2-4cfe-a7d0-3b17e038fbb5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:02:05.967793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3ec1260e-48f2-4cfe-a7d0-3b17e038fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:05.976538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 18:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:02:06.129537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c155cf6-4ae0-4e7a-bcbe-bef13f962ff2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:02:06.143728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c155cf6-4ae0-4e7a-bcbe-bef13f962ff2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:02:06.149417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0ee80-6873-4f91-9af9-c27dbe1e157d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:02:06.150010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0ee80-6873-4f91-9af9-c27dbe1e157d service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:06.150288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0ee80-6873-4f91-9af9-c27dbe1e157d service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:06.150560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0ee80-6873-4f91-9af9-c27dbe1e157d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:06.158886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0ee80-6873-4f91-9af9-c27dbe1e157d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:02:06.161015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0ee80-6873-4f91-9af9-c27dbe1e157d service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:02:06.170286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 766/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:10.025731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0b323-908d-4636-814f-2be2d8f0a9ca None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:10.026279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0b323-908d-4636-814f-2be2d8f0a9ca None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:10.026279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0b323-908d-4636-814f-2be2d8f0a9ca None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:10.027623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 18:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:02:10.033223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c081e-095f-459f-b00d-42ed85877544 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:10.033630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c081e-095f-459f-b00d-42ed85877544 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:10.033792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c081e-095f-459f-b00d-42ed85877544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:10.054062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f68c081e-095f-459f-b00d-42ed85877544 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:02:10.055259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f68c081e-095f-459f-b00d-42ed85877544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:10.063120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:02:10.355335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0b323-908d-4636-814f-2be2d8f0a9ca None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:02:10.357111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a5a0b323-908d-4636-814f-2be2d8f0a9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:10.436680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0b323-908d-4636-814f-2be2d8f0a9ca None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:02:10.440190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc55d7-cfac-4c0e-adf8-4743b2fd5daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:02:10.440811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc55d7-cfac-4c0e-adf8-4743b2fd5daf service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:10.441113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc55d7-cfac-4c0e-adf8-4743b2fd5daf service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:10.441357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc55d7-cfac-4c0e-adf8-4743b2fd5daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:10.450104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc55d7-cfac-4c0e-adf8-4743b2fd5daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:02:10.451537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc55d7-cfac-4c0e-adf8-4743b2fd5daf service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:02:10.461785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 763/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:12.351739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d860814-4dcb-453c-ae21-14cf988c57c4 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:12.352297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d860814-4dcb-453c-ae21-14cf988c57c4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:12.352297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d860814-4dcb-453c-ae21-14cf988c57c4 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:12.353671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 18:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:02:12.358794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba9107-e7d5-4e5b-9dbe-e56295805c35 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:12.359257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba9107-e7d5-4e5b-9dbe-e56295805c35 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:12.359358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba9107-e7d5-4e5b-9dbe-e56295805c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:12.379290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d9ba9107-e7d5-4e5b-9dbe-e56295805c35 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:02:12.380504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d9ba9107-e7d5-4e5b-9dbe-e56295805c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:12.388578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:02:12.674719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d860814-4dcb-453c-ae21-14cf988c57c4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:02:12.686201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d860814-4dcb-453c-ae21-14cf988c57c4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:02:12.689761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ed0b-1e78-464e-87f2-b3c24c5a93e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:02:12.690361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ed0b-1e78-464e-87f2-b3c24c5a93e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:12.690644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ed0b-1e78-464e-87f2-b3c24c5a93e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:12.691487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ed0b-1e78-464e-87f2-b3c24c5a93e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:12.701270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ed0b-1e78-464e-87f2-b3c24c5a93e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:02:12.702734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ed0b-1e78-464e-87f2-b3c24c5a93e3 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:02:12.712771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 769/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:16.298823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f83536-e369-45e7-af6f-bbeb01003927 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:16.299334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f83536-e369-45e7-af6f-bbeb01003927 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:16.299447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f83536-e369-45e7-af6f-bbeb01003927 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:16.301325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 18:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:02:16.306976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5716b58-f00d-4a8c-9caf-53ccf54be10f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:16.307398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5716b58-f00d-4a8c-9caf-53ccf54be10f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:16.307531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5716b58-f00d-4a8c-9caf-53ccf54be10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:16.328857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a5716b58-f00d-4a8c-9caf-53ccf54be10f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:02:16.330041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a5716b58-f00d-4a8c-9caf-53ccf54be10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:16.337715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 18:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:02:16.895478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f83536-e369-45e7-af6f-bbeb01003927 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:02:16.897240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-79f83536-e369-45e7-af6f-bbeb01003927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:16.961885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:02:16.975419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f83536-e369-45e7-af6f-bbeb01003927 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:02:16.978976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e36de5-8780-472e-beb1-30967e9088d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:02:16.979571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e36de5-8780-472e-beb1-30967e9088d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:16.979850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e36de5-8780-472e-beb1-30967e9088d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:16.980117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e36de5-8780-472e-beb1-30967e9088d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:16.989032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e36de5-8780-472e-beb1-30967e9088d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:02:16.990792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e36de5-8780-472e-beb1-30967e9088d6 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:02:17.000225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 766/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:18.922316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4070f-7165-455c-8a30-abc045ff1e71 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:18.922876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4070f-7165-455c-8a30-abc045ff1e71 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:18.922876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4070f-7165-455c-8a30-abc045ff1e71 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:18.924194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 18:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:02:18.929364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6ccbf-deb4-42c7-8391-715a31895124 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:18.929748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6ccbf-deb4-42c7-8391-715a31895124 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:18.929917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6ccbf-deb4-42c7-8391-715a31895124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:18.949607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-80b6ccbf-deb4-42c7-8391-715a31895124 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:02:18.950801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-80b6ccbf-deb4-42c7-8391-715a31895124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:18.958778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 18:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:02:19.512824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4070f-7165-455c-8a30-abc045ff1e71 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:02:19.514475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-48f4070f-7165-455c-8a30-abc045ff1e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:02:19.527463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4070f-7165-455c-8a30-abc045ff1e71 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:02:19.531213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685477-172b-4406-845c-8185316531f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:02:19.531796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685477-172b-4406-845c-8185316531f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:02:19.532072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685477-172b-4406-845c-8185316531f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:02:19.532369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685477-172b-4406-845c-8185316531f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:02:19.541809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685477-172b-4406-845c-8185316531f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:02:19.543202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685477-172b-4406-845c-8185316531f6 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:02:19.553154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 772/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:02:28.944368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=11 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:02:29.548388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:44.229897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.231818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.231818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.261435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:44.265245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:44.278781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 768/1540] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:44.288734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.289195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.289286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.319901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:44.321150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:44.339366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 773/1541] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:44.343868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.357705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d8a6c-091c-43af-b4c1-f9d2838e2da3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.366358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.367039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.367332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.367607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.368274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:44.368609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.370274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.392434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 769/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:03:44.401263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.422431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6932be-ed73-463a-8e7a-58cc78268001 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.432563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8488c-63f4-4048-88f0-5ea9288fcad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.433200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8488c-63f4-4048-88f0-5ea9288fcad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.433481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8488c-63f4-4048-88f0-5ea9288fcad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.433777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8488c-63f4-4048-88f0-5ea9288fcad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.434450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8488c-63f4-4048-88f0-5ea9288fcad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.440604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8488c-63f4-4048-88f0-5ea9288fcad3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.546263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 774/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:44.550250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.565553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec659d-3a4d-4bcd-a9d2-131b7c20f245 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.648144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.648807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.649089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.649359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.650012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.652036 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.793582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86e95-c2da-467c-af9f-1276101dda90 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.793582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86e95-c2da-467c-af9f-1276101dda90 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.793582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86e95-c2da-467c-af9f-1276101dda90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.794078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7be86e95-c2da-467c-af9f-1276101dda90 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:44.794946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7be86e95-c2da-467c-af9f-1276101dda90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:44.807794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 775/1544] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:44.825388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.825388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.825388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.847343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:44.849151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:44.860197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 770/1545] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:44.861225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 776/1546] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:03:44.864059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.867402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.877632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc8f73-8784-481d-937d-978a7c9301b0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.887030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce991924-85d0-40df-bad6-34548a70fe64 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.889463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.890187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.890493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.890759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.891537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:44.891850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.892983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:03:44.892983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:03:44.892983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:03:44.893983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.901079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.901710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.901996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.902276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.902953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.904642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.912592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 777/1547] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:03:44.922582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 771/1548] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:44.926968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.927139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:44.942777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b5aec-3b15-4e64-9737-b7081fbecae6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.944628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc721-6b54-48b5-afba-499ebeeca82e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:44.950932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.951553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.951853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.952114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.952782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.954789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.961012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:44.961632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:44.961907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:44.962170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:44.962837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:44.964410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:44.992175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 772/1549] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:03:45.002370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.021643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05c11-6e48-44b7-971a-1d1f52fa3547 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.034199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.034914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.035211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.035489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] PATH_INFO: `/v3/projects/6aae701e8155487b8a50d7300642ee56/users/e26cbd3dcec948f69529a9541bb33ce1/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.067308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aae701e8155487b8a50d7300642ee56, user_id=e26cbd3dcec948f69529a9541bb33ce1, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.067308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:03:45.067308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:03:45.067308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:03:45.067308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d63d7-b8d9-44a9-a1c7-c41cab1d2837 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.097941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 778/1550] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:44 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:45.102580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.174531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 773/1551] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:45 2022] PUT /identity/v3/projects/6aae701e8155487b8a50d7300642ee56/users/e26cbd3dcec948f69529a9541bb33ce1/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:03:45.175445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4684695-8c82-45ee-b940-0120998a9bcd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.175890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298ddb-e824-45c6-bc81-d285c424d826 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.176231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298ddb-e824-45c6-bc81-d285c424d826 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.177189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298ddb-e824-45c6-bc81-d285c424d826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.205182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-eb298ddb-e824-45c6-bc81-d285c424d826 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.206402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-eb298ddb-e824-45c6-bc81-d285c424d826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.234614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.235236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.235516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.235789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.236462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.237469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:03:45.237469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:03:45.237469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:03:45.238824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9093d-96d8-44a2-b57c-02d9335c8263 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.268748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 774/1552] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.270611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3e7f6-c291-4c83-8e02-8a0403a95819 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.270901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3e7f6-c291-4c83-8e02-8a0403a95819 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.271184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3e7f6-c291-4c83-8e02-8a0403a95819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.291624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6bb3e7f6-c291-4c83-8e02-8a0403a95819 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.292867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6bb3e7f6-c291-4c83-8e02-8a0403a95819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.300925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 775/1553] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.302282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8482bd-ce30-4d11-838c-c08b623571ab None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.302565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8482bd-ce30-4d11-838c-c08b623571ab None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.302840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8482bd-ce30-4d11-838c-c08b623571ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.321679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6f8482bd-ce30-4d11-838c-c08b623571ab None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.322797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6f8482bd-ce30-4d11-838c-c08b623571ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.330740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 776/1554] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.331898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57346a-e037-4e65-b9a8-8011e011dcbc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.332184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57346a-e037-4e65-b9a8-8011e011dcbc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.332495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57346a-e037-4e65-b9a8-8011e011dcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.343780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 779/1555] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:03:45.345732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4705c-9a22-4a26-96c8-91771824bbf7 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.346196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4705c-9a22-4a26-96c8-91771824bbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.346676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4705c-9a22-4a26-96c8-91771824bbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.353082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-af57346a-e037-4e65-b9a8-8011e011dcbc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.354326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-af57346a-e037-4e65-b9a8-8011e011dcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.362434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 777/1556] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.363633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8306905-4907-4b95-a1b6-a8dd03c16d1e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.364607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8306905-4907-4b95-a1b6-a8dd03c16d1e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.364915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8306905-4907-4b95-a1b6-a8dd03c16d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.373299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:45.374802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-28d4705c-9a22-4a26-96c8-91771824bbf7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.376038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-28d4705c-9a22-4a26-96c8-91771824bbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.384748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e8306905-4907-4b95-a1b6-a8dd03c16d1e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.385083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 780/1557] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.386425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.386710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.386984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.389537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e8306905-4907-4b95-a1b6-a8dd03c16d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.394930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:45.398054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 778/1558] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.399282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d4e07-8661-4fdf-ab51-1ba04ac6053e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.399822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d4e07-8661-4fdf-ab51-1ba04ac6053e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.399949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d4e07-8661-4fdf-ab51-1ba04ac6053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.416059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] MFA Rules not processed for user `e26cbd3dcec948f69529a9541bb33ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.417315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.421560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ae1d4e07-8661-4fdf-ab51-1ba04ac6053e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.422774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ae1d4e07-8661-4fdf-ab51-1ba04ac6053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.431427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 779/1559] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.434262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51208-9549-447d-944d-c83d05c6b0f0 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.434262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51208-9549-447d-944d-c83d05c6b0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.434262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51208-9549-447d-944d-c83d05c6b0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.452577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-55e51208-9549-447d-944d-c83d05c6b0f0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.453759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-55e51208-9549-447d-944d-c83d05c6b0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.461995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 780/1560] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.463306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138a9c6-ca16-4386-b57f-df18373be061 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.463588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138a9c6-ca16-4386-b57f-df18373be061 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.463857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138a9c6-ca16-4386-b57f-df18373be061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.484674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c138a9c6-ca16-4386-b57f-df18373be061 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.485285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 781/1561] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.485977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.486097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c138a9c6-ca16-4386-b57f-df18373be061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.496364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc9dd-f552-4bb3-8ce5-c11abc6ff286 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.497694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 781/1562] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.501448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33990345-29d8-4b7c-bddd-60f5be12889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.502061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990345-29d8-4b7c-bddd-60f5be12889b admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.502374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990345-29d8-4b7c-bddd-60f5be12889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.502648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990345-29d8-4b7c-bddd-60f5be12889b admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.502853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194d61c-4936-45ef-9d74-f09c2232a510 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.503162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194d61c-4936-45ef-9d74-f09c2232a510 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.503283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990345-29d8-4b7c-bddd-60f5be12889b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.503434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194d61c-4936-45ef-9d74-f09c2232a510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.505524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990345-29d8-4b7c-bddd-60f5be12889b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.517599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 782/1563] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:45.519000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.519317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.519565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.523889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e194d61c-4936-45ef-9d74-f09c2232a510 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.525237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e194d61c-4936-45ef-9d74-f09c2232a510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.537866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 782/1564] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.539174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.539460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.539735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.543779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.545019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.553839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 783/1565] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.554537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.560057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.571198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.571328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9a1f3-46f3-4bca-b3e0-b5ef43285bac None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.571328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.571328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.571609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 783/1566] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.571731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.571731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.571905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.572001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:45.572001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.572467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.580065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb6b73-0c5b-49eb-9e44-6f5758bd058f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.591148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 784/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:45.591328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.591470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.591589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.591589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.591589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.591589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:45.591589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.591589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.598005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b07f-4279-4f84-81d9-e283f935ce5d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.601424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 784/1568] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:45.602229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.603677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.604286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.604602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.604938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.605594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:45.605903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.607487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345024d-fa7f-4c57-a770-15eb620c7f5b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.612495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa4c3c-967e-4e02-be26-3fb9290fb7e1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.617932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.618525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.618802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.619072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.619713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.621309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.622974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 785/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:45.623113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775839d1-e479-41af-99a7-7e40050c91ad None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.623374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775839d1-e479-41af-99a7-7e40050c91ad None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.623649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775839d1-e479-41af-99a7-7e40050c91ad None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.624904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 786/1570] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 1 18:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:03:45.625604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775839d1-e479-41af-99a7-7e40050c91ad None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.641417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 785/1571] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:45.641648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775839d1-e479-41af-99a7-7e40050c91ad None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.641768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.643044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.643710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.644064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.644380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.645039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:45.645453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.647061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.649389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe33623-f457-4f82-83d2-b8ca123bcc60 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.653893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.654477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.654754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.655023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.655929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:45.656228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.657815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.663718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 787/1572] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:45.663718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.670003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4224f-5c2c-464f-9f8f-4cb6cac36972 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.674599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.674748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 786/1573] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:45.675415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.681164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.681571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.682490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.683166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:45.683475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.685136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.685789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eb7d5-2da4-43e3-a8ca-92d95c336f52 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.690925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.691522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.691806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.692204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.693393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.694409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:03:45.694409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:03:45.694409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:03:45.695595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234a00f-8e5d-43d5-b776-36d3d0022e23 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.704814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 788/1574] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:45.704814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.710180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2b9fc-fc3a-4a8d-959d-6fd5f4c460dc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.715314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.715902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.716337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.716629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.717256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.719184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.779990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 787/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:45.781691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.782042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.782311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.818244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 789/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:45.819259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.826772 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:45.828161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:45.893341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009030d-782f-45bd-84c2-17f664504407 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.898786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.899242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 788/1577] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:45.899471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.899760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.900079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:45.900205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.900947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.902877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:45.910076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:45.911501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8893b-83f7-45de-b29c-812025066e74 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:45.916130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:45.916822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:45.917101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:45.917376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] PATH_INFO: `/v3/projects/62c83a25c5f74e3396ebe8b0a3ee580e/users/af540344a27b48eba16db2861df2078b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:45.933003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c83a25c5f74e3396ebe8b0a3ee580e, user_id=af540344a27b48eba16db2861df2078b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:45.935718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.024173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 789/1578] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:45 2022] PUT /identity/v3/projects/62c83a25c5f74e3396ebe8b0a3ee580e/users/af540344a27b48eba16db2861df2078b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:46.025018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.029830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 790/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:45 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.029830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.050132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:46.090611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45652e3-d678-4c35-9694-98545a43571c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.091850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33b347-0fd4-4485-8371-f09ce860fd89 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.095225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.095838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.096122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.096435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.096831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.097066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.097436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.097721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.097997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.099478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.099743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.101482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.205317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 791/1580] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/projects => generated 393 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.205317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.219430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 790/1581] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.220118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.268466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac2de-bcdf-4ef5-b3bf-b3ddd71bcb06 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.284950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914eb7-ff1f-4067-a2f0-3fd9b8816503 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.289823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.290533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.290810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.291079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.291712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.293677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.326464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.327068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.327346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.327616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.328239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.330122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.396144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 791/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/users => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.397125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.406951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e23142-d4df-4c2d-807f-7aed091f3f51 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.411634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99f31-2f95-442b-a937-1c640df177f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.412206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99f31-2f95-442b-a937-1c640df177f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.412513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99f31-2f95-442b-a937-1c640df177f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.412784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99f31-2f95-442b-a937-1c640df177f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.413399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99f31-2f95-442b-a937-1c640df177f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.415284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99f31-2f95-442b-a937-1c640df177f0 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.425537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 792/1583] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.427288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.432627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:46.496289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 792/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.496289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.496289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.496699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45763e-5e2c-4432-951d-4ee3ac54cabd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.496791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.501762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.502504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.502827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.503101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.515383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:46.516628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:46.562165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.563737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.573315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 793/1585] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:46.574074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.581245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 793/1586] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:46.581245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.588287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa981a-c45d-406a-81d8-4c3ef13c5ffc service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.591448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684cddd-e7cc-4853-812e-a7eab9c586ce None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.593521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.594198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.594542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.594876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.595589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.597195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.597666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.597965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.598523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.599001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.599914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.601982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.664449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 794/1587] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.665249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.675579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc3e4e-e1eb-405d-814a-616501f7f332 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.678846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:46.681640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.682008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.682291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.682560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.683179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.688337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 794/1588] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.688493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.688582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.696846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fa390-94cb-4d69-933f-405ea7d3ae96 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.701757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.702395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.702679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.702953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.703625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.705251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.717456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 795/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.718306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.728327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014d830-6869-48ef-857d-900d62e7cd91 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.732966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.733535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.733807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.734075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.734683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.736569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.798768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 795/1590] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.799655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.810337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57101258-fae2-4d19-a2ca-ba5c9bd18d22 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.815527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.816127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.816561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.816866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.818142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.819723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.822819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 796/1591] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:46 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:46.825653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.831742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 796/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.832532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.842114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff0f2-542a-4fee-82d7-6b2323539375 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.846955 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.847614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.847898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.848196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.848880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.850443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.860266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 797/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.861083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.871007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27beb-c5aa-4581-b2c2-e13d1fca274b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.875981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.876698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.877032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.877306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.877948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.880096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.881997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4403ae-47de-4648-ac67-415af05e01e6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.886047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.886282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.886559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.886829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.890572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 798/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.894339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.897115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.898554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.903755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b588a-0dea-4746-bac7-33c6f266f786 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.908380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 797/1595] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:46.908775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0e38-7147-4a0a-a5ed-562b83701f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.909165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.909417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0e38-7147-4a0a-a5ed-562b83701f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.909815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0e38-7147-4a0a-a5ed-562b83701f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.910085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0e38-7147-4a0a-a5ed-562b83701f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.911301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0e38-7147-4a0a-a5ed-562b83701f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.913132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0e38-7147-4a0a-a5ed-562b83701f17 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.919604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50df8b-015a-4410-8783-6ebe851bac30 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.923813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 799/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.925068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c35a4-b945-4f63-a8f3-8fef5a3bd273 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.925412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c35a4-b945-4f63-a8f3-8fef5a3bd273 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.925686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c35a4-b945-4f63-a8f3-8fef5a3bd273 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.928160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 800/1597] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 1 18:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:03:46.928920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c35a4-b945-4f63-a8f3-8fef5a3bd273 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.931324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.932063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.933093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.933378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.934069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.935675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.941686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c35a4-b945-4f63-a8f3-8fef5a3bd273 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.946774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 798/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.946904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.951709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.951709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.951892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.951980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.951980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.951980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.957797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43294663-4ada-4989-9c12-0d298ae9f30c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.962475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.963141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.963283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 801/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.964034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.966354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.966645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.967309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.968939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.974522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6559e-d7ca-4883-acd6-d7d719ff1a6e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.979583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:46.979726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 799/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.980231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:46.980524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:46.980632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:46.980932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:46.981552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:46.983182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:46.989271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:46.992929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd609-6786-4669-a0d4-460eff392410 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:46.995075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 802/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:46.995899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.000789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18515593-95cb-465c-ba4b-3b94cb0f2c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.001422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515593-95cb-465c-ba4b-3b94cb0f2c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.001703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515593-95cb-465c-ba4b-3b94cb0f2c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.001975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515593-95cb-465c-ba4b-3b94cb0f2c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.002618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18515593-95cb-465c-ba4b-3b94cb0f2c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.004239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18515593-95cb-465c-ba4b-3b94cb0f2c0d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.006751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d7e6-a376-4a55-bfa3-28c7ddf45926 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.011711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.012380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.012661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.013020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] PATH_INFO: `/v3/projects/48c5a892321046ffb6cc2b772c19a126/users/11fcf1f255ab4abb9c08ee05d2aab4eb/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.016740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 800/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:47.017769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.018056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.018327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.033981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c5a892321046ffb6cc2b772c19a126, user_id=11fcf1f255ab4abb9c08ee05d2aab4eb, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.037160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.039387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:47.040649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:47.053970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 801/1603] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:47.061330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.073111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc75cd-4662-464b-8387-1c885adeb2f5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.078216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.082151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.082151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.082151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.082151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.082151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.096024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 802/1604] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:47.096024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.106385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee58f81-9d5f-4210-9698-683425b3ffb6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.116740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 803/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:46 2022] PUT /identity/v3/projects/48c5a892321046ffb6cc2b772c19a126/users/11fcf1f255ab4abb9c08ee05d2aab4eb/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:47.116740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.171190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.171840 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.172187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.172496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.173147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.174774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.183208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d96c-35f9-4f64-a384-5b7885ef30d6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.185595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 803/1606] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:47.185595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.193676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.194572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.195126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.195570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] PATH_INFO: `/v3/projects/7f83480b456d4c9f8fe4a74f3b81f6e3/users/4911fa3dc5264eceb36f2601830cca81/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.199241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94ac94-4f0c-41c6-a353-baf1c04d1354 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.204562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.205229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.205503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.205781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] PATH_INFO: `/v3/projects/b40443d415fa40338b837dd445032342/users/fb6a740de8b9417b89afb08f230aa4b7/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.213646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f83480b456d4c9f8fe4a74f3b81f6e3, user_id=4911fa3dc5264eceb36f2601830cca81, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.216686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.222305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40443d415fa40338b837dd445032342, user_id=fb6a740de8b9417b89afb08f230aa4b7, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.225573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.229241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:47.303295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 804/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:47 2022] PUT /identity/v3/projects/7f83480b456d4c9f8fe4a74f3b81f6e3/users/4911fa3dc5264eceb36f2601830cca81/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:47.304533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.327067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 804/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:47 2022] PUT /identity/v3/projects/b40443d415fa40338b837dd445032342/users/fb6a740de8b9417b89afb08f230aa4b7/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:47.328226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.378231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23befb81-7eee-44de-ba89-c2a3f5492582 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.386577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83af030-41b5-4c39-912f-373fcfadab5e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.390496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.391113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.391398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.391680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.450325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c337b-3a58-4635-a9fe-43d0bb359a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.451029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c337b-3a58-4635-a9fe-43d0bb359a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.451266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c337b-3a58-4635-a9fe-43d0bb359a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.451541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c337b-3a58-4635-a9fe-43d0bb359a33 admin admin] PATH_INFO: `/v3/projects/6749f998c37a43428eb7e1233408c560/users/01f1499bbf89441dae30433aaf1d29a7/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.463235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:47.470969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c337b-3a58-4635-a9fe-43d0bb359a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6749f998c37a43428eb7e1233408c560, user_id=01f1499bbf89441dae30433aaf1d29a7, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.470969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c337b-3a58-4635-a9fe-43d0bb359a33 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.471177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.471718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.483127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 805/1609] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:47.484090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.495332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c562c-1307-4845-8119-3b6010769833 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.500963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.501904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.502257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.502610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] PATH_INFO: `/v3/projects/f7d9509296034fda840fc3b3548bfb5c/users/6f40b97920434d1492030ac0f4d89ffd/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.528864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d9509296034fda840fc3b3548bfb5c, user_id=6f40b97920434d1492030ac0f4d89ffd, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.531910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.562704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 805/1610] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:47 2022] PUT /identity/v3/projects/6749f998c37a43428eb7e1233408c560/users/01f1499bbf89441dae30433aaf1d29a7/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:47.564008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794a39f-26f9-43de-9ca2-c28ea79ca9a8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.568632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794a39f-26f9-43de-9ca2-c28ea79ca9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.568632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794a39f-26f9-43de-9ca2-c28ea79ca9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.605650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9794a39f-26f9-43de-9ca2-c28ea79ca9a8 None None] MFA Rules not processed for user `11fcf1f255ab4abb9c08ee05d2aab4eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:47.605650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9794a39f-26f9-43de-9ca2-c28ea79ca9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:47.664455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 806/1611] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:47 2022] PUT /identity/v3/projects/f7d9509296034fda840fc3b3548bfb5c/users/6f40b97920434d1492030ac0f4d89ffd/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:47.665552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.682565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 806/1612] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:47.684049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871171c-2ce3-44e2-9e35-15c6eb84bf4e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.684349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871171c-2ce3-44e2-9e35-15c6eb84bf4e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.684630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871171c-2ce3-44e2-9e35-15c6eb84bf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.714069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a871171c-2ce3-44e2-9e35-15c6eb84bf4e None None] MFA Rules not processed for user `4911fa3dc5264eceb36f2601830cca81`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:47.715268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a871171c-2ce3-44e2-9e35-15c6eb84bf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:47.762015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da297a-e7e6-426c-aa88-8dfbf79df05a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:47.769981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:47.770784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.771058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.771397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] PATH_INFO: `/v3/projects/bf4415a1201b45e6b27db755a90aef07/users/a4eef62b9b6a4d92a9945db556b54910/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.795684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4415a1201b45e6b27db755a90aef07, user_id=a4eef62b9b6a4d92a9945db556b54910, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:47.800645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:47.802531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:47.806897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 807/1613] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:47.808727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc2a2-2ee7-47de-9eac-7479980b49dc None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.809015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc2a2-2ee7-47de-9eac-7479980b49dc None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.809307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbc2a2-2ee7-47de-9eac-7479980b49dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.831156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-21fbc2a2-2ee7-47de-9eac-7479980b49dc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:47.832646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-21fbc2a2-2ee7-47de-9eac-7479980b49dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:47.842478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 808/1614] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:47.848046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b95ffe-64d9-49ad-9071-c94c84025459 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.848046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b95ffe-64d9-49ad-9071-c94c84025459 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.848046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b95ffe-64d9-49ad-9071-c94c84025459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.868644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-62b95ffe-64d9-49ad-9071-c94c84025459 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:47.869829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-62b95ffe-64d9-49ad-9071-c94c84025459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:47.882879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 809/1615] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:47.884225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0d492-796a-4964-a4c2-56df26327c23 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.884567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0d492-796a-4964-a4c2-56df26327c23 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.884843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0d492-796a-4964-a4c2-56df26327c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.917218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-8eb0d492-796a-4964-a4c2-56df26327c23 None None] MFA Rules not processed for user `6f40b97920434d1492030ac0f4d89ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:47.918450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8eb0d492-796a-4964-a4c2-56df26327c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:47.956063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 807/1616] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:47 2022] PUT /identity/v3/projects/bf4415a1201b45e6b27db755a90aef07/users/a4eef62b9b6a4d92a9945db556b54910/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:47.956063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.967118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:47.994232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 810/1617] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:47.994232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b975-940d-48db-a924-c158c0b326a0 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:47.994232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b975-940d-48db-a924-c158c0b326a0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:47.994232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b975-940d-48db-a924-c158c0b326a0 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:47.994232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 811/1618] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 1 18:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:03:47.995033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8b975-940d-48db-a924-c158c0b326a0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:47.996795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-95f8b975-940d-48db-a924-c158c0b326a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:48.020713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc65c2e-272b-436b-b1a7-d9935d7658bb admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.026699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc282b-415d-4037-afe4-ab35169a4375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.027488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc282b-415d-4037-afe4-ab35169a4375 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.027954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc282b-415d-4037-afe4-ab35169a4375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.028285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc282b-415d-4037-afe4-ab35169a4375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.029422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc282b-415d-4037-afe4-ab35169a4375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.031386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc282b-415d-4037-afe4-ab35169a4375 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.077236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8b975-940d-48db-a924-c158c0b326a0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.081074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.081789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.082167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.083123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.092667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.104233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.104393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 812/1619] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:48.120908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.126808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 808/1620] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:47 2022] POST /identity/v3/projects => generated 403 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:48.126808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5151aff-e61b-4832-8af8-1d00ee6396b6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.126808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5151aff-e61b-4832-8af8-1d00ee6396b6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.126808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5151aff-e61b-4832-8af8-1d00ee6396b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.130101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b0288-ecd9-46fe-a523-ea3a3112a65b service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.155786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c5151aff-e61b-4832-8af8-1d00ee6396b6 None None] MFA Rules not processed for user `a4eef62b9b6a4d92a9945db556b54910`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:48.157038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c5151aff-e61b-4832-8af8-1d00ee6396b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:48.183463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.184222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.184524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.184796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.224806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 809/1621] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:48.226003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.226286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.226554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.252353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:48.252561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.252662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:48.252757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.260970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 813/1622] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:48.261890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.346863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51665653-47b5-4e63-8763-7fbe86062fab service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.354909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.356637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 810/1623] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 1 18:03:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:48.357141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.357141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.357310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.357552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.368515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:48.383159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a007e2d-4bd5-4bdf-8884-0f21731e4ee9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.391247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.392194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.393181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.393777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.394675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.397795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.425113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.427627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.445431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 814/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:48.446091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.480410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe8c95-217a-48bc-bd3b-891132c511e3 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.487935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.489631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.490005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.490490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.491246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.494067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.545929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 811/1625] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:48 2022] POST /identity/v3/users => generated 308 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:48.549604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.578435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c40b2d-ef84-4e4b-a9cd-28544cb0fa06 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.589730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.590552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.590834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.591156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.651840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 815/1626] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:48 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:48.652930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.731211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:48.739603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.750441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.788197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cacc-6b04-48c1-962c-931fd66a5c21 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.789145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 812/1627] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:48.789492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.790210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.790562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.790970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.791849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.793695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.794126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.822875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 816/1628] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:48.847652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7234a4-08be-4b94-8d7c-7858aa78ac78 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.853373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:48.870869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.872775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.872775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.872775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.874373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.876360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.876694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31aed-18d4-4541-9080-1c8ba0ccfd72 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:48.903697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:48.905129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:48.905689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:48.905980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] PATH_INFO: `/v3/projects/833d173dac784ba5bcd66deae41c0e3f/users/e23b9ecde896462ba4a2ec5f5463885f/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:48.973413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=833d173dac784ba5bcd66deae41c0e3f, user_id=e23b9ecde896462ba4a2ec5f5463885f, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:48.994086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:48.994285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 813/1629] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:48 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:48.994285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.008997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26aca3-cffd-4441-af36-e5b9099c3214 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.022868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.023500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.023777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.024051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.024861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.026483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.039777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 814/1630] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:03:49.044251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.064631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1999130-653f-4123-b032-1a45a8893de0 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.076582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.077275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.077607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.077884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] PATH_INFO: `/v3/projects/d9aef57f76104c02bf04daacd3c50b1b/users/9ec34c46b70f49dc8a87e2bc59034fad/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.120006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9aef57f76104c02bf04daacd3c50b1b, user_id=9ec34c46b70f49dc8a87e2bc59034fad, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.126835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.151763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 817/1631] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:48 2022] PUT /identity/v3/projects/833d173dac784ba5bcd66deae41c0e3f/users/e23b9ecde896462ba4a2ec5f5463885f/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:49.156864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.239979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 815/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:49 2022] PUT /identity/v3/projects/d9aef57f76104c02bf04daacd3c50b1b/users/9ec34c46b70f49dc8a87e2bc59034fad/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:49.243210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.245179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:49.267601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ff59d-3bbb-466b-b2e4-72f8224367ff admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.356388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9871e-2a02-41ab-8ab2-9f9595146df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.356388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9871e-2a02-41ab-8ab2-9f9595146df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.356388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9871e-2a02-41ab-8ab2-9f9595146df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.356388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9871e-2a02-41ab-8ab2-9f9595146df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.358364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9871e-2a02-41ab-8ab2-9f9595146df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.358872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9871e-2a02-41ab-8ab2-9f9595146df8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.372163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 818/1633] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:49.373776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece63c3-379e-4ce7-b391-7571d24b54f4 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.374068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece63c3-379e-4ce7-b391-7571d24b54f4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.374660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece63c3-379e-4ce7-b391-7571d24b54f4 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.376911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 819/1634] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 1 18:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:03:49.377774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece63c3-379e-4ce7-b391-7571d24b54f4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.386259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfefb2-04d7-42c4-bd5b-63112e989719 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.393793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece63c3-379e-4ce7-b391-7571d24b54f4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.399753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e80fd6-754b-4630-9908-d053d6bc32d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.400647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e80fd6-754b-4630-9908-d053d6bc32d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.400967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e80fd6-754b-4630-9908-d053d6bc32d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.401248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e80fd6-754b-4630-9908-d053d6bc32d5 admin admin] PATH_INFO: `/v3/projects/833d173dac784ba5bcd66deae41c0e3f/users/e23b9ecde896462ba4a2ec5f5463885f/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.403766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e80fd6-754b-4630-9908-d053d6bc32d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833d173dac784ba5bcd66deae41c0e3f, user_id=e23b9ecde896462ba4a2ec5f5463885f, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.404535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72d633-2b91-47bb-9a7f-95a8fdcaedab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.406190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72d633-2b91-47bb-9a7f-95a8fdcaedab admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.406684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72d633-2b91-47bb-9a7f-95a8fdcaedab admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.407146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72d633-2b91-47bb-9a7f-95a8fdcaedab admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.407743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e80fd6-754b-4630-9908-d053d6bc32d5 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.408392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72d633-2b91-47bb-9a7f-95a8fdcaedab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.414815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72d633-2b91-47bb-9a7f-95a8fdcaedab admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.421394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 820/1635] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:49 2022] PUT /identity/v3/projects/833d173dac784ba5bcd66deae41c0e3f/users/e23b9ecde896462ba4a2ec5f5463885f/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:49.422614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.422901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.423224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.440862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 816/1636] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:49.442906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.444023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.447608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.453063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:49.455738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:49.516821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] MFA Rules not processed for user `e23b9ecde896462ba4a2ec5f5463885f`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:49.518012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:49.535007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:49.601369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 821/1637] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:49.607633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.625738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 817/1638] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:49.627018 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.704674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48db66-a55e-4cba-ac66-d72f01d0a423 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.705318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2950a-9857-4fcc-909a-97c4c759982f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.709249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.710077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.710423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.710645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.712191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dfe01-cc54-45a9-ae66-fb51a8072885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:49.713119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dfe01-cc54-45a9-ae66-fb51a8072885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.713323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dfe01-cc54-45a9-ae66-fb51a8072885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.713603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dfe01-cc54-45a9-ae66-fb51a8072885 admin admin] PATH_INFO: `/v3/projects/d9aef57f76104c02bf04daacd3c50b1b/users/9ec34c46b70f49dc8a87e2bc59034fad/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.715545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126dfe01-cc54-45a9-ae66-fb51a8072885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9aef57f76104c02bf04daacd3c50b1b, user_id=9ec34c46b70f49dc8a87e2bc59034fad, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.718383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126dfe01-cc54-45a9-ae66-fb51a8072885 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.725070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:49.725421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:49.727390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:49.732361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 822/1639] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:49 2022] PUT /identity/v3/projects/d9aef57f76104c02bf04daacd3c50b1b/users/9ec34c46b70f49dc8a87e2bc59034fad/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:49.741959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f0421-aa40-450f-831d-da420303b352 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:49.742315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f0421-aa40-450f-831d-da420303b352 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:49.742649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f0421-aa40-450f-831d-da420303b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:49.747646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 818/1640] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:49.798555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-866f0421-aa40-450f-831d-da420303b352 None None] MFA Rules not processed for user `9ec34c46b70f49dc8a87e2bc59034fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:49.800670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-866f0421-aa40-450f-831d-da420303b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:49.963215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:49.972506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382116-f89c-4e0b-b811-229b2f677c6c service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:49.996816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 823/1641] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:50.065901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:50.066524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:50.067367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:50.067647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:50.216806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:50.216806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:50.507976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 819/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:49 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:50.565910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f0421-aa40-450f-831d-da420303b352 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:50.587989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f0421-aa40-450f-831d-da420303b352 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:50.594472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:50.595380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:50.598090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:50.598090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:50.612692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:50.614937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:50.638443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 824/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:50.986884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:51.057850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:51.121845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a249f3-25ba-4ee6-95ae-176f4e8a07df service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:51.132990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:51.133651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:51.135362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:51.137882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] PATH_INFO: `/v3/users/a4eef62b9b6a4d92a9945db556b54910` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:51.137882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4eef62b9b6a4d92a9945db556b54910)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:51.139193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: WARNING py.warnings [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:03:51.139193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: warnings.warn(msg) Sep 01 18:03:51.139193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: Sep 01 18:03:51.145588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:51.242343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:51.263520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68cd91-8a0b-46c4-ba0e-6d87f4529267 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:51.281317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:51.281977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:51.282291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:51.282569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] PATH_INFO: `/v3/users/11fcf1f255ab4abb9c08ee05d2aab4eb` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:51.289779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fcf1f255ab4abb9c08ee05d2aab4eb)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:51.295744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:51.516203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4eef62b9b6a4d92a9945db556b54910'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.541448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4eef62b9b6a4d92a9945db556b54910'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.654413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fcf1f255ab4abb9c08ee05d2aab4eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.657873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4eef62b9b6a4d92a9945db556b54910'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.659046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4eef62b9b6a4d92a9945db556b54910'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.669103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fcf1f255ab4abb9c08ee05d2aab4eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.669999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fcf1f255ab4abb9c08ee05d2aab4eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.671436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fcf1f255ab4abb9c08ee05d2aab4eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.729176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d990ada4-5567-40ae-a0ad-b2a5cbd74092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4eef62b9b6a4d92a9945db556b54910'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.749175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 820/1644] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:03:50 2022] DELETE /identity/v3/users/a4eef62b9b6a4d92a9945db556b54910 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:51.756061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:51.756524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:51.756920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:51.761017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-751bc2f3-fbec-4b8d-848a-3091a329240e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fcf1f255ab4abb9c08ee05d2aab4eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:51.836996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 825/1645] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:03:51 2022] DELETE /identity/v3/users/11fcf1f255ab4abb9c08ee05d2aab4eb => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:03:51.848015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:51.850713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:51.851446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:51.860340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:51.862188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:51.909915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:51.911220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:52.010238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:52.063202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 821/1646] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:52.066307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 826/1647] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:52.104965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:52.106825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:52.117321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:52.119539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:52.180831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:52.270497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524baf8-f0a8-4544-a862-e6a6fe4a724a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:52.298072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:52.300993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:52.301879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:52.302317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:52.308167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1870d68-3bfc-4ee7-802d-2d55dc70323e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:52.331342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:52.332064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:52.332463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:52.336566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:52.344458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:52.346251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:52.347204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:52.353740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:52.370189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 827/1648] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:52.380066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 822/1649] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:03:53.680527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:53.681444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:53.704849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fab41-d8dd-4332-b87b-fb72ac83fcab service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:53.719350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:53.720354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:53.720354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:53.720610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] PATH_INFO: `/v3/projects/bf4415a1201b45e6b27db755a90aef07` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:53.721780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4415a1201b45e6b27db755a90aef07)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:53.723872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: WARNING py.warnings [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 18:03:53.723872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: warnings.warn(msg) Sep 01 18:03:53.723872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: Sep 01 18:03:53.725276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:53.993622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4415a1201b45e6b27db755a90aef07', 'request_id': 'req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.059740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4415a1201b45e6b27db755a90aef07', 'request_id': 'req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.071262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4415a1201b45e6b27db755a90aef07', 'request_id': 'req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.119621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] The token cache is being invalidate because project bf4415a1201b45e6b27db755a90aef07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:03:54.119935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.121022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.126943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 828/1650] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:03:53 2022] DELETE /identity/v3/projects/bf4415a1201b45e6b27db755a90aef07 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:54.131863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:54.146176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff837a23-628c-413a-be00-b54819426b5e service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:54.197699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:54.199178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:54.374529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:54.377325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:54.377758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:54.378178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] PATH_INFO: `/v3/users/9ec34c46b70f49dc8a87e2bc59034fad` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:54.382269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec34c46b70f49dc8a87e2bc59034fad)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:54.386111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:54.438212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cce38-4a66-4e16-9686-b4b6ebd13b18 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:54.450175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:54.473169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:54.473169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:54.473169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] PATH_INFO: `/v3/projects/48c5a892321046ffb6cc2b772c19a126` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:54.473169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c5a892321046ffb6cc2b772c19a126)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:54.473169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:54.497872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:54.718584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec34c46b70f49dc8a87e2bc59034fad'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.735724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec34c46b70f49dc8a87e2bc59034fad'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.750400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c5a892321046ffb6cc2b772c19a126', 'request_id': 'req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.815070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec34c46b70f49dc8a87e2bc59034fad'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.818435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec34c46b70f49dc8a87e2bc59034fad'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.838673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec34c46b70f49dc8a87e2bc59034fad'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.868264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c5a892321046ffb6cc2b772c19a126', 'request_id': 'req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.875796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 823/1651] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:03:54 2022] DELETE /identity/v3/users/9ec34c46b70f49dc8a87e2bc59034fad => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:54.884252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c5a892321046ffb6cc2b772c19a126', 'request_id': 'req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.916981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] The token cache is being invalidate because project 48c5a892321046ffb6cc2b772c19a126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:03:54.917396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.919668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:54.924524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 829/1652] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:03:54 2022] DELETE /identity/v3/projects/48c5a892321046ffb6cc2b772c19a126 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:03:54.933493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:54.935710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:55.042781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:55.065471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edaa2ea-3b8c-48a5-ac82-f4fee3a3b8e1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:55.071506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:55.072657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:55.073279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:55.073747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] PATH_INFO: `/v3/users/e23b9ecde896462ba4a2ec5f5463885f` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:55.075948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] RBAC: Authorizing `identity:delete_user(user_id=e23b9ecde896462ba4a2ec5f5463885f)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:55.078192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:55.411963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e23b9ecde896462ba4a2ec5f5463885f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:55.432892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e23b9ecde896462ba4a2ec5f5463885f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:55.541997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e23b9ecde896462ba4a2ec5f5463885f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:55.543303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e23b9ecde896462ba4a2ec5f5463885f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:55.591418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e23b9ecde896462ba4a2ec5f5463885f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:55.626916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 824/1653] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:03:54 2022] DELETE /identity/v3/users/e23b9ecde896462ba4a2ec5f5463885f => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:03:56.719513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:56.721865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:56.872549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f5bf7-bcc4-463a-9416-00de3ed83e69 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:56.999341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:57.000945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:57.008031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:57.009145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.009658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.010158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] PATH_INFO: `/v3/projects/833d173dac784ba5bcd66deae41c0e3f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.013280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833d173dac784ba5bcd66deae41c0e3f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:57.017280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:57.021516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67055a3-ca34-4014-bd29-90c55cc1afde admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:57.040280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:57.041005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.041442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.041880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] PATH_INFO: `/v3/projects/d9aef57f76104c02bf04daacd3c50b1b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.043439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9aef57f76104c02bf04daacd3c50b1b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:57.048459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:57.234433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9aef57f76104c02bf04daacd3c50b1b', 'request_id': 'req-43fce9bb-931b-4049-9318-1fd301eb7208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.262361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:57.267162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833d173dac784ba5bcd66deae41c0e3f', 'request_id': 'req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.315516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9aef57f76104c02bf04daacd3c50b1b', 'request_id': 'req-43fce9bb-931b-4049-9318-1fd301eb7208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.325632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9aef57f76104c02bf04daacd3c50b1b', 'request_id': 'req-43fce9bb-931b-4049-9318-1fd301eb7208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.350992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] The token cache is being invalidate because project d9aef57f76104c02bf04daacd3c50b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:03:57.351318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.356238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-43fce9bb-931b-4049-9318-1fd301eb7208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.358373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 825/1654] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:03:56 2022] DELETE /identity/v3/projects/d9aef57f76104c02bf04daacd3c50b1b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:03:57.364303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d3af6-a263-4856-9c5c-49d8c6d3ab5d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.364823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d3af6-a263-4856-9c5c-49d8c6d3ab5d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.365282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d3af6-a263-4856-9c5c-49d8c6d3ab5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.405023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833d173dac784ba5bcd66deae41c0e3f', 'request_id': 'req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.413628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833d173dac784ba5bcd66deae41c0e3f', 'request_id': 'req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.416971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7d2d3af6-a263-4856-9c5c-49d8c6d3ab5d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:57.419982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7d2d3af6-a263-4856-9c5c-49d8c6d3ab5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:57.446122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] The token cache is being invalidate because project 833d173dac784ba5bcd66deae41c0e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:03:57.446763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.447653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4f22a44a-2fd8-4f3a-8929-4a384690cbd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:03:57.452995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 830/1655] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:03:56 2022] DELETE /identity/v3/projects/833d173dac784ba5bcd66deae41c0e3f => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:57.455435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535e76d-38da-4c73-a117-a5ca5049c597 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.455926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535e76d-38da-4c73-a117-a5ca5049c597 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.456439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535e76d-38da-4c73-a117-a5ca5049c597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.509147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c535e76d-38da-4c73-a117-a5ca5049c597 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:57.511690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c535e76d-38da-4c73-a117-a5ca5049c597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:57.552854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:57.609883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 826/1656] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:03:57.620081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.620559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.620838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.672175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 831/1657] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:57.674659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.675191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.675669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.718312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:57.719600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:57.730908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 827/1658] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:03:57.735330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:57.740609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:57.743149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:57.747362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d209c-4b98-49a3-ae57-2df2c2d56ab1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:57.767492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:57.768672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.769019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.769497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.770273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:57.770667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:57.772547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:57.791965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 832/1659] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:57.795459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:57.801581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 828/1660] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:03:57.802783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:57.814764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49686b-ad77-4c38-877c-7287d901c013 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:57.819505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3802d-e6d7-4708-95c9-f6c6294edcbb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:57.820659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:57.821625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.824449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.825012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.825845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:57.826250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:57.826877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.827157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.827427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.828426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:03:57.828729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:57.830243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:57.830656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:57.868524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 833/1661] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:03:57.873042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:57.906067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a211887-82bc-46ed-ba62-21c840f0c825 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:57.921463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:57.922402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:57.922905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:57.925106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:57.927466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:57.932545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:57.971465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 829/1662] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/projects => generated 399 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:57.973076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.050742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1b8cb-3943-4eef-8576-8ca3d67db020 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.073935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.076182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.076844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.077227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.080230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.084650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.133054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 834/1663] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/projects => generated 371 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:58.134758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.240246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 830/1664] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:57 2022] POST /identity/v3/users => generated 300 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:03:58.240246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.262063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d1272-b782-403c-b51e-9a8618737eab admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.268632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42cc21-558c-4912-b69f-7b7ec266bc9d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.285024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.286202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.289141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.289141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.289141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.289662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.292398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.293196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.293513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.295935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.298237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.308254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.351574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 831/1665] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:03:58.352847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.377759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba3a6-188e-4afc-a295-fc2e7f194993 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.408354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.409130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.409419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.409697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.414132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.419124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.468916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 832/1666] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:58.474433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.481844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 835/1667] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:58 2022] POST /identity/v3/users => generated 286 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:58.485885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.508620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd561b-34b4-4f93-b905-86c9fb6c4594 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.525303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.526944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.538709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.538709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.539366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.541045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.541541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7bb2-e6cd-4f25-820c-3ed130900d66 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.572229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044844fe-6b55-4db8-a573-99d00baaf316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.579086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044844fe-6b55-4db8-a573-99d00baaf316 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.580388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044844fe-6b55-4db8-a573-99d00baaf316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.581449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044844fe-6b55-4db8-a573-99d00baaf316 admin admin] PATH_INFO: `/v3/projects/34d77d812f2a422cbc65b8c6e633bd98/users/ef37438d7e0945babf1ec23bb9fd9ccf/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.619520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 836/1668] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:03:58.620593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.621641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044844fe-6b55-4db8-a573-99d00baaf316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d77d812f2a422cbc65b8c6e633bd98, user_id=ef37438d7e0945babf1ec23bb9fd9ccf, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.627871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044844fe-6b55-4db8-a573-99d00baaf316 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.650021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a4e6b-b6a3-4557-a747-afeb4eab0200 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.659105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.659745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.660033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.660974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.661569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:58.663140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:58.696347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 837/1669] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:03:58.700417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:58.735828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 833/1670] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:58 2022] PUT /identity/v3/projects/34d77d812f2a422cbc65b8c6e633bd98/users/ef37438d7e0945babf1ec23bb9fd9ccf/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:58.747392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.748299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.749636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:58.754408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba0052-2b4a-48ca-a537-3869af65d69d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:58.834909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] MFA Rules not processed for user `ef37438d7e0945babf1ec23bb9fd9ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:58.844434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:58.944213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94635364-b00e-4ef2-8895-3e3bcca354b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:58.949114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635364-b00e-4ef2-8895-3e3bcca354b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:58.949388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635364-b00e-4ef2-8895-3e3bcca354b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:58.949676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635364-b00e-4ef2-8895-3e3bcca354b0 admin admin] PATH_INFO: `/v3/projects/ab7e06dcbf3442ca98787f5201ddad64/users/1e2be61ea35a47269c0454f023bbbf20/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.011011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94635364-b00e-4ef2-8895-3e3bcca354b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7e06dcbf3442ca98787f5201ddad64, user_id=1e2be61ea35a47269c0454f023bbbf20, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:59.014123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94635364-b00e-4ef2-8895-3e3bcca354b0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:59.094689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 834/1671] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:03:59.099493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:59.114576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c13ea7-a76c-417c-a1fe-f57aac2d7b7c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:59.120857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041f594-2221-432c-9697-5f1f139cf8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:59.121676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041f594-2221-432c-9697-5f1f139cf8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.123055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041f594-2221-432c-9697-5f1f139cf8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.123055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041f594-2221-432c-9697-5f1f139cf8b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.123613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041f594-2221-432c-9697-5f1f139cf8b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:59.126287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041f594-2221-432c-9697-5f1f139cf8b5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:59.204454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 838/1672] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:03:58 2022] PUT /identity/v3/projects/ab7e06dcbf3442ca98787f5201ddad64/users/1e2be61ea35a47269c0454f023bbbf20/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:03:59.208554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.208554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.214297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.265495 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] MFA Rules not processed for user `af540344a27b48eba16db2861df2078b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:59.267142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:59.277807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 835/1673] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/projects => generated 401 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:59.278654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff461245-0d5c-4384-ab41-9e3457e0e5d2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.278945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff461245-0d5c-4384-ab41-9e3457e0e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.279237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff461245-0d5c-4384-ab41-9e3457e0e5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.293739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:59.352593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:03:59.356137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ff461245-0d5c-4384-ab41-9e3457e0e5d2 None None] MFA Rules not processed for user `1e2be61ea35a47269c0454f023bbbf20`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:59.357650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ff461245-0d5c-4384-ab41-9e3457e0e5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:59.484284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 839/1674] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:59.487543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:59.554060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 836/1675] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:59.558129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0230bb2-b590-4d60-bcf6-ece434b8e85b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.558477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0230bb2-b590-4d60-bcf6-ece434b8e85b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.558752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0230bb2-b590-4d60-bcf6-ece434b8e85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.599348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a0230bb2-b590-4d60-bcf6-ece434b8e85b None None] MFA Rules not processed for user `fb6a740de8b9417b89afb08f230aa4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:59.600813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a0230bb2-b590-4d60-bcf6-ece434b8e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:59.651030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81920e-ee97-493b-aad9-0b4bfb7b95f2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:59.707778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 837/1676] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:59.709344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.709961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.710532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.781497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] MFA Rules not processed for user `af540344a27b48eba16db2861df2078b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:03:59.785711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:59.797464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:59.799033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.799033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.799033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:03:59.812958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:03:59.815343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:03:59.935958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 840/1677] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/users => generated 307 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:03:59.936735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:59.959330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 838/1678] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:03:59.960378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:03:59.967568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:03:59.983584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426a99-c248-4dc2-9f87-d5d804a1494c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:03:59.993764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:03:59.994386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:03:59.994666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:03:59.994939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.000109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.005731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.148159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 841/1679] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:03:59 2022] POST /identity/v3/projects => generated 373 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:00.149669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:00.202252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2371cb-9052-45e2-80b2-b3270134b0a2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:00.311200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcfa20-202b-41e1-9ecb-3d83ab535b88 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:00.325138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:00.341834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:00.342486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.342766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.343039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.343707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.345335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.380218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 842/1680] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:00.381143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:00.385194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:00.401862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:00.402500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.402778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.403056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.425241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af026-b0f4-4338-9757-1e0d2bf91430 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:00.451400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:00.452122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.452431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.452706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] PATH_INFO: `/v3/users/4911fa3dc5264eceb36f2601830cca81` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.454600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4911fa3dc5264eceb36f2601830cca81)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.456678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.547115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.549146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.559329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 839/1681] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:03:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:00.560058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:00.561304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:00.576965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bea5-9ef4-4649-9051-5af93408609e service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:00.584557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:00.585254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.585604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.585948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.673944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.675600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.685942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 840/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:00.686862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:00.698830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f447b5a-e474-499e-a367-c1a4cb5c5fb5 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:00.705637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:00.706288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.706659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.706933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.708354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.710291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.791175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4911fa3dc5264eceb36f2601830cca81'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:00.810867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:00.812738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 841/1683] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:00 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:00.813654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:00.833742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b43d92-765d-4e3c-a3f2-147fe0b7e395 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:00.844289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4911fa3dc5264eceb36f2601830cca81'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:00.847248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4911fa3dc5264eceb36f2601830cca81'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:00.848284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4911fa3dc5264eceb36f2601830cca81'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:00.854686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238e6e9-6561-46a0-bffa-c582e6cf8896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:00.855835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238e6e9-6561-46a0-bffa-c582e6cf8896 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.856100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238e6e9-6561-46a0-bffa-c582e6cf8896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.856473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238e6e9-6561-46a0-bffa-c582e6cf8896 admin admin] PATH_INFO: `/v3/projects/94db85b0d5bd453f8c1d3c2afb35ec56/users/e4ec01feb76542b8b9812ce818327ecf/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:00.877151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238e6e9-6561-46a0-bffa-c582e6cf8896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94db85b0d5bd453f8c1d3c2afb35ec56, user_id=e4ec01feb76542b8b9812ce818327ecf, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:00.881040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238e6e9-6561-46a0-bffa-c582e6cf8896 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:00.972580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4911fa3dc5264eceb36f2601830cca81'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:00.990805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 842/1684] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:00 2022] PUT /identity/v3/projects/94db85b0d5bd453f8c1d3c2afb35ec56/users/e4ec01feb76542b8b9812ce818327ecf/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:00.992548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:00.993108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:00.993509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.009087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 843/1685] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:00 2022] DELETE /identity/v3/users/4911fa3dc5264eceb36f2601830cca81 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:01.010167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.012799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.081399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] MFA Rules not processed for user `01f1499bbf89441dae30433aaf1d29a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:01.089003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.164113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 843/1686] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:01.165360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.167043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.190384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10fda-796e-47de-8c2d-332ca0c386d7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.207544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b054-3af1-436e-90e2-ca6f0fd63038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.207544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b054-3af1-436e-90e2-ca6f0fd63038 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.207544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b054-3af1-436e-90e2-ca6f0fd63038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.207544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b054-3af1-436e-90e2-ca6f0fd63038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.208194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793b054-3af1-436e-90e2-ca6f0fd63038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.208194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793b054-3af1-436e-90e2-ca6f0fd63038 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.218430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 844/1687] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:01.220357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.220654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.221230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.233533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1d90-b8a4-43f5-a7ac-0cc65e6f727b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.256539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba076b7b-cd79-4915-acf6-5d5742249f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.257201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba076b7b-cd79-4915-acf6-5d5742249f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.257750 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba076b7b-cd79-4915-acf6-5d5742249f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.258335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba076b7b-cd79-4915-acf6-5d5742249f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.260097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba076b7b-cd79-4915-acf6-5d5742249f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.262148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba076b7b-cd79-4915-acf6-5d5742249f6a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.270349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:01.276156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 844/1688] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:01.278535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.279034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.279486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.280195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.298539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 845/1689] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:01.300025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.387392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060a1f-b72c-41a2-8e51-06378111a0ab None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.387488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] MFA Rules not processed for user `01f1499bbf89441dae30433aaf1d29a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:01.387488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.387488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 845/1690] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:01.387488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.387732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374cac-4c6a-4f6b-a4ab-d33599309fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.387732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374cac-4c6a-4f6b-a4ab-d33599309fff admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.387732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374cac-4c6a-4f6b-a4ab-d33599309fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.387732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374cac-4c6a-4f6b-a4ab-d33599309fff admin admin] PATH_INFO: `/v3/projects/23f17661792e458a8a6078130961618c/users/76c61940b7ee42a1b927bf768eac4473/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.387992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96791428-4714-45e9-97d1-63f6860eb30c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.388062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374cac-4c6a-4f6b-a4ab-d33599309fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f17661792e458a8a6078130961618c, user_id=76c61940b7ee42a1b927bf768eac4473, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.388136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.388136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.388278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374cac-4c6a-4f6b-a4ab-d33599309fff admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.388377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.388377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] PATH_INFO: `/v3/projects/94db85b0d5bd453f8c1d3c2afb35ec56/users/e4ec01feb76542b8b9812ce818327ecf/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.388377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94db85b0d5bd453f8c1d3c2afb35ec56, user_id=e4ec01feb76542b8b9812ce818327ecf, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.388377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.388377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 846/1691] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:01 2022] PUT /identity/v3/projects/94db85b0d5bd453f8c1d3c2afb35ec56/users/e4ec01feb76542b8b9812ce818327ecf/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:01.388677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.391751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.465907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14033669-92dd-4f27-94ff-aa6737580883 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.474045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.474045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.474680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.475039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.525332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 846/1692] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:01 2022] PUT /identity/v3/projects/23f17661792e458a8a6078130961618c/users/76c61940b7ee42a1b927bf768eac4473/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:01.526403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32713ba7-4fa0-4802-94eb-4cf411fdb9a3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.526694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32713ba7-4fa0-4802-94eb-4cf411fdb9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.527001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32713ba7-4fa0-4802-94eb-4cf411fdb9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.557896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.559810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.572547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 847/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:01.573521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.574902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.585119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-32713ba7-4fa0-4802-94eb-4cf411fdb9a3 None None] MFA Rules not processed for user `e4ec01feb76542b8b9812ce818327ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:01.586331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-32713ba7-4fa0-4802-94eb-4cf411fdb9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.669305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad06e1-2ac5-479a-b4f9-bbf875fdc259 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.708844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 847/1694] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:01.716283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666604-b531-4bfe-9468-38cc35bc5917 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.718806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666604-b531-4bfe-9468-38cc35bc5917 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.719290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666604-b531-4bfe-9468-38cc35bc5917 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.720961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 848/1695] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 1 18:04:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:04:01.729127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49666604-b531-4bfe-9468-38cc35bc5917 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.797648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b85e2-1033-41f6-8ad6-e32410b220da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.798291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b85e2-1033-41f6-8ad6-e32410b220da service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.798628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b85e2-1033-41f6-8ad6-e32410b220da service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.798911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b85e2-1033-41f6-8ad6-e32410b220da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.801838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:01.823952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49666604-b531-4bfe-9468-38cc35bc5917 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.831558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.834294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.834586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.834864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.835732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.838756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.863937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 849/1696] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:01.863937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:01.864361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:01.892391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b85e2-1033-41f6-8ad6-e32410b220da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.892391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b85e2-1033-41f6-8ad6-e32410b220da service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.907770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 848/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:01.911063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.911596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.912178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.946233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab69b7c-ecb6-4b89-bd24-73c2d0c573dc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:01.951683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:01.952572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:01.952855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:01.953137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:01.970422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:01.972581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:01.996149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:01.997908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:02.016023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 850/1698] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:02.016023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:02.034311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e9700-a558-47aa-9bbb-b22746fb9054 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:02.050687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e343ee-9f06-4558-95ac-9db2df24bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:02.052259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e343ee-9f06-4558-95ac-9db2df24bbb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:02.052770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e343ee-9f06-4558-95ac-9db2df24bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:02.053057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e343ee-9f06-4558-95ac-9db2df24bbb9 admin admin] PATH_INFO: `/v3/projects/23f17661792e458a8a6078130961618c/users/76c61940b7ee42a1b927bf768eac4473/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:02.058834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e343ee-9f06-4558-95ac-9db2df24bbb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f17661792e458a8a6078130961618c, user_id=76c61940b7ee42a1b927bf768eac4473, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:02.064079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e343ee-9f06-4558-95ac-9db2df24bbb9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:02.082846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 851/1699] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:02 2022] PUT /identity/v3/projects/23f17661792e458a8a6078130961618c/users/76c61940b7ee42a1b927bf768eac4473/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:02.094074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:02.094530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:02.095656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:02.106427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 849/1700] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:02.131627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:02.133279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:02.153547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8ee7e-21ce-4af9-a006-5c3112714785 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:02.178290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] MFA Rules not processed for user `76c61940b7ee42a1b927bf768eac4473`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:02.186287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:02.223306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:02.224095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:02.224538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:02.224910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:02.239280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:02.239280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:02.253076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 850/1701] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:02.312011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 852/1702] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:02.322442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:02.394346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:02.397371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:02.430491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:02.466283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff70261-3d88-4e95-9846-11a815bde948 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:02.471333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:02.472113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:02.472528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:02.473645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:02.494657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:02.496428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:02.525272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 851/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:02.557075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:02.607748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af118c-4090-4d98-bed8-f3c34ce3c4e8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:02.608167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:02.609328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:02.609617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:02.609901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:02.615273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:02.751426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:02.777373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:02.794549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 853/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:02.891444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:02.897658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:02.900176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:02.914407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9795ee-a1d9-40df-916f-3ca68c511058 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:02.930620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:02.931396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:02.931951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:02.932234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:02.939094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.023822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fe28a-b5c6-4e91-8a0c-2f002b671429 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:03.042224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:03.043809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.045261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.045687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] PATH_INFO: `/v3/projects/7f83480b456d4c9f8fe4a74f3b81f6e3` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.048158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f83480b456d4c9f8fe4a74f3b81f6e3)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:03.054896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:03.123673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:03.125301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeea984-fb83-4150-9e90-004da2fc7b6c service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:03.137224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 852/1705] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:03.269087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f83480b456d4c9f8fe4a74f3b81f6e3', 'request_id': 'req-5c555011-684a-4ca1-8812-9849f2dadb91'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:03.350451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d95a6-32bc-405c-bcf6-b2bd086af560 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.350925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d95a6-32bc-405c-bcf6-b2bd086af560 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.351184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d95a6-32bc-405c-bcf6-b2bd086af560 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.353148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 853/1706] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 1 18:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 18:04:03.360198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.360198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.360198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.377984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f83480b456d4c9f8fe4a74f3b81f6e3', 'request_id': 'req-5c555011-684a-4ca1-8812-9849f2dadb91'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:03.384678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f83480b456d4c9f8fe4a74f3b81f6e3', 'request_id': 'req-5c555011-684a-4ca1-8812-9849f2dadb91'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:03.388040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:03.389426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.425886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] The token cache is being invalidate because project 7f83480b456d4c9f8fe4a74f3b81f6e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:03.426258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:03.436463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c555011-684a-4ca1-8812-9849f2dadb91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:03.438048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 854/1707] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:02 2022] DELETE /identity/v3/projects/7f83480b456d4c9f8fe4a74f3b81f6e3 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:03.439563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0847a3b-d777-40ed-96b1-c43fff9adf22 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.439857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0847a3b-d777-40ed-96b1-c43fff9adf22 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.440148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0847a3b-d777-40ed-96b1-c43fff9adf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.515774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e0847a3b-d777-40ed-96b1-c43fff9adf22 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:03.517085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e0847a3b-d777-40ed-96b1-c43fff9adf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.525927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 854/1708] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:03.534679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:03.536642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.603682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d9ee9-a35d-46e0-b387-9dc0f998b15a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:03.603682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:03.603682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.603682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.604077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.637514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 855/1709] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:03.639243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:03.639894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.640191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.640497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.643540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:03.673352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 855/1710] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:03.676122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:03.677991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:03.680043 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.680172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.699271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 856/1711] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:03.706754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:03.729308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01207b13-66ad-492a-84ab-44d9dafda1f4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:03.742236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:03.742809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.743095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.743369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.744039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:03.744357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:03.746028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:03.778913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 857/1712] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:03.783856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:03.806742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec65ba-767c-4f23-ab22-86f0881e578d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:03.817684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:03.819463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.819755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.820050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.820743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:03.828125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:03.845631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cfcd5-c9ef-47a7-80ac-7aaf361b180a service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:03.857970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:03.858602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:03.858883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:03.859156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:03.864968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:03.882434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:03.884136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:03.921714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 856/1713] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:03.957648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 858/1714] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:03 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:03.961851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:03.978476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e6668-aa64-4583-94ec-9037e4fb9db4 service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:04.011949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:04.198012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:04.198648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:04.198928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:04.199198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:04.200116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:04.204121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:04.410732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 857/1715] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:03 2022] POST /identity/v3/users => generated 287 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:04.414476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:04.428467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912db711-b10c-4167-aa0f-5264f4330329 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:04.440525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:04.440525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:04.440525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:04.440525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:04.442038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:04.442038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:04.455505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 859/1716] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:04.456773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:04.476892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8dfe-5c74-4d88-bb29-560ee39f982d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:04.510345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e47fc2-8d3b-4336-b2c0-b8f30243867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:04.511229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e47fc2-8d3b-4336-b2c0-b8f30243867f admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:04.511706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e47fc2-8d3b-4336-b2c0-b8f30243867f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:04.512154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e47fc2-8d3b-4336-b2c0-b8f30243867f admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:04.513229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e47fc2-8d3b-4336-b2c0-b8f30243867f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:04.520651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e47fc2-8d3b-4336-b2c0-b8f30243867f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:04.553081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:04.567233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 858/1717] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:04.567857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61936e4-655b-4b85-88d6-c9ccf1d2f810 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:04.584044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:04.585056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:04.585555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:04.586472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] PATH_INFO: `/v3/projects/d0164a8160f8461988819f82476c4b8d/users/a1788edcc6ef4fa8a75e1b932ced7a7b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:04.636816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0164a8160f8461988819f82476c4b8d, user_id=a1788edcc6ef4fa8a75e1b932ced7a7b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:04.639610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:04.744992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 860/1718] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:04 2022] PUT /identity/v3/projects/d0164a8160f8461988819f82476c4b8d/users/a1788edcc6ef4fa8a75e1b932ced7a7b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:04.757460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3e59-5d83-4375-ab28-fda11093b837 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:04.757798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3e59-5d83-4375-ab28-fda11093b837 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:04.758759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a3e59-5d83-4375-ab28-fda11093b837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:04.768140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:04.770761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:04.786274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:04.845605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-089a3e59-5d83-4375-ab28-fda11093b837 None None] MFA Rules not processed for user `a1788edcc6ef4fa8a75e1b932ced7a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:04.845605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-089a3e59-5d83-4375-ab28-fda11093b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:04.951817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dd948-d4b7-41f5-9853-ec49c18b380b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:04.951817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:04.951817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:04.951817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:04.952482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] PATH_INFO: `/v3/users/e26cbd3dcec948f69529a9541bb33ce1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:04.954248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26cbd3dcec948f69529a9541bb33ce1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:04.957248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:05.019349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 859/1719] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:05.021725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:05.022013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:05.022287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:05.059903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:05.089385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] MFA Rules not processed for user `01f1499bbf89441dae30433aaf1d29a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:05.089385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:05.227006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26cbd3dcec948f69529a9541bb33ce1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:05.252306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26cbd3dcec948f69529a9541bb33ce1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:05.256464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26cbd3dcec948f69529a9541bb33ce1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:05.257390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 860/1720] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:05.258483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:05.259707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26cbd3dcec948f69529a9541bb33ce1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:05.262648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:05.361418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-65ec9538-f4bc-4712-a4ca-20c980a08c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26cbd3dcec948f69529a9541bb33ce1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:05.436532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 861/1721] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:04 2022] DELETE /identity/v3/users/e26cbd3dcec948f69529a9541bb33ce1 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:05.438213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:05.438504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:05.438779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:05.489141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a92043-3e78-4745-91d3-62103eb74f60 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:05.509019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:05.516926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:05.670156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:05.670835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:05.671119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:05.671454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:05.673433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:05.715535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 862/1722] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:05.722013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:05.723803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:05.834579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:05.837252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b799cf6-8a61-4576-a3c6-823e9847348b service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:05.858852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 861/1723] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:05.951908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:05.966228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea8608-de3e-4e1a-9b9d-a3b5ba74e499 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:05.984888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:05.984888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:05.984888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:05.984888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:06.042454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7158fd-0213-40f5-941d-66c1e234c41c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:06.042746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7158fd-0213-40f5-941d-66c1e234c41c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:06.043027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7158fd-0213-40f5-941d-66c1e234c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:06.050650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:06.065878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:06.069396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6b7158fd-0213-40f5-941d-66c1e234c41c None None] MFA Rules not processed for user `fb6a740de8b9417b89afb08f230aa4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:06.071082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6b7158fd-0213-40f5-941d-66c1e234c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:06.131311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 863/1724] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:06.220474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 862/1725] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:06.497364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:07.094330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:07.281229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:07.281229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:07.301068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e6266-037d-439e-bdad-b6c9473ce6fa service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:07.325000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:07.325700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:07.325933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:07.326207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] PATH_INFO: `/v3/projects/6aae701e8155487b8a50d7300642ee56` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:07.344115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aae701e8155487b8a50d7300642ee56)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:07.344115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:07.596551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aae701e8155487b8a50d7300642ee56', 'request_id': 'req-1385268e-9e26-4da4-a62a-8aea726106e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:07.719256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aae701e8155487b8a50d7300642ee56', 'request_id': 'req-1385268e-9e26-4da4-a62a-8aea726106e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:07.726326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aae701e8155487b8a50d7300642ee56', 'request_id': 'req-1385268e-9e26-4da4-a62a-8aea726106e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:07.757627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] The token cache is being invalidate because project 6aae701e8155487b8a50d7300642ee56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:07.757917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:07.769505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1385268e-9e26-4da4-a62a-8aea726106e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:07.769505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 864/1726] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:07 2022] DELETE /identity/v3/projects/6aae701e8155487b8a50d7300642ee56 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:07.772176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:07.772707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:07.773128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:07.848810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:07.850264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:07.943374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 863/1727] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:07.951013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:07.951324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:07.951605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:07.975815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:07.976777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:07.991307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 865/1728] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:07.998279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.009723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc1bae-2d6b-4f50-854e-631427134402 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.015530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.016147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.016613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.016931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.017865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:08.018815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.022545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:08.045540 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.055446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 864/1729] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:08.060711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33856e14-decd-41af-bc81-11c36bd0c0af None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.084617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.085857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.086362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.086639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.089033 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.096137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:08.281511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.282500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 866/1730] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:08 2022] POST /identity/v3/projects => generated 369 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:08.308649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b89cb-9dc6-49f9-9c2f-ac1c6b0ac1d4 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.491420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.492185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.492498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.492943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.493859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.495779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:08.671852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 865/1731] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:08 2022] POST /identity/v3/users => generated 285 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:08.672636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.728491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd453704-5e04-48c6-9442-af0a39331c08 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.751748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.755492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.755492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.755492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.755492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.756002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:08.777862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 867/1732] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:08.779295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.793960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbf416-0643-4981-bec4-37e0a0bf6281 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.802261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.803356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.803813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.804247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.805177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.807509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:08.835014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 866/1733] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:08.838157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.888887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f1f0-e7d4-43eb-8045-45d9960df4fa admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.891649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:08.893612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b554cf7-5696-4879-a264-4417316c8e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.893755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:08.895688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b554cf7-5696-4879-a264-4417316c8e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.896187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b554cf7-5696-4879-a264-4417316c8e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.897252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b554cf7-5696-4879-a264-4417316c8e97 admin admin] PATH_INFO: `/v3/projects/6c7163d42a4f48a59e46564e7d8b0fb4/users/3781b1652ea84bb6bf8ca1f6bf02b1bb/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.913593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e63877-4438-4f83-8e06-ce18ded56c11 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:08.918748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:08.919390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:08.921071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:08.931288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] PATH_INFO: `/v3/users/a1788edcc6ef4fa8a75e1b932ced7a7b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:08.931288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1788edcc6ef4fa8a75e1b932ced7a7b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.945106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:08.954691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b554cf7-5696-4879-a264-4417316c8e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7163d42a4f48a59e46564e7d8b0fb4, user_id=3781b1652ea84bb6bf8ca1f6bf02b1bb, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:08.961893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b554cf7-5696-4879-a264-4417316c8e97 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:09.017343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:09.087709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 868/1734] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:08 2022] PUT /identity/v3/projects/6c7163d42a4f48a59e46564e7d8b0fb4/users/3781b1652ea84bb6bf8ca1f6bf02b1bb/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:09.107069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.109519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.109519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.179576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] MFA Rules not processed for user `3781b1652ea84bb6bf8ca1f6bf02b1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:09.181075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:09.197827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1788edcc6ef4fa8a75e1b932ced7a7b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:09.206739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1788edcc6ef4fa8a75e1b932ced7a7b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:09.272045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 869/1735] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:09.277988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:09.327417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1788edcc6ef4fa8a75e1b932ced7a7b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:09.329180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1788edcc6ef4fa8a75e1b932ced7a7b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:09.343523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-821073d1-711d-4d4a-bde5-545f174b1516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1788edcc6ef4fa8a75e1b932ced7a7b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:09.355123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96040a-b9d3-4a83-b13e-5df51ff0ba26 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:09.358199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 867/1736] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:08 2022] DELETE /identity/v3/users/a1788edcc6ef4fa8a75e1b932ced7a7b => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:09.362789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.363796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.364206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.371603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:09.390679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:09.391976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:09.450435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:09.451115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.451604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.451887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.453507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:09.458829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:09.462432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 868/1737] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:09.470943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:09.474208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:09.532712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c90920-e864-4d53-80b5-6cd8709d88ae None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:09.536845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:09.537496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.537838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.538212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.547841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:09.549569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:09.561185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 869/1738] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:09.592046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 870/1739] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:09 2022] POST /identity/v3/projects => generated 369 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:09.596566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:09.598065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:09.611367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7941f-c855-48d9-b94b-ab84b7c1f8a2 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:09.693327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:09.694017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.694304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.694578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.695215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:09.697259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:09.790626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 870/1740] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:09 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:09.794666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:09.816907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dd025-d025-41bf-a770-3960a3b05434 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:09.816907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:09.816907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.816907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.817257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.817257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:09.818064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:09.830714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 871/1741] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:09.836026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:09.846369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2228a-c7c7-4055-9264-f38d0758f9f7 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:09.852010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:09.852762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:09.853114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:09.853459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] PATH_INFO: `/v3/projects/239f48cf6aea468eaf63125493a56cdf/users/178bea54a3aa4ff39f3d665f6a0149e1/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:09.869663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=239f48cf6aea468eaf63125493a56cdf, user_id=178bea54a3aa4ff39f3d665f6a0149e1, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:09.872766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.010558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 871/1742] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:09 2022] PUT /identity/v3/projects/239f48cf6aea468eaf63125493a56cdf/users/178bea54a3aa4ff39f3d665f6a0149e1/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:10.014420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.025961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654fb65-f89d-4e3d-ae01-9bb2df914d7d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.083844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa463d-60cb-4889-a5d8-1d4c3243cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.084477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa463d-60cb-4889-a5d8-1d4c3243cec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.084757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa463d-60cb-4889-a5d8-1d4c3243cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.085028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa463d-60cb-4889-a5d8-1d4c3243cec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.085670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa463d-60cb-4889-a5d8-1d4c3243cec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.087236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa463d-60cb-4889-a5d8-1d4c3243cec9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.098331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 872/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:10.102316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.112486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a495456-02ce-4c84-9e1b-8e28c8332cab admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.116995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.117641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.127726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.127726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] PATH_INFO: `/v3/projects/239f48cf6aea468eaf63125493a56cdf/users/178bea54a3aa4ff39f3d665f6a0149e1/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.127726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239f48cf6aea468eaf63125493a56cdf, user_id=178bea54a3aa4ff39f3d665f6a0149e1, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.127726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.133426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 872/1744] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:10 2022] PUT /identity/v3/projects/239f48cf6aea468eaf63125493a56cdf/users/178bea54a3aa4ff39f3d665f6a0149e1/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:10.143480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.143766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.144042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.172048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] MFA Rules not processed for user `178bea54a3aa4ff39f3d665f6a0149e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:10.173276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.189228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.197683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.268672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 873/1745] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:10.278276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.279677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.287193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f81f-885f-4b2c-98f8-dc263e9dd570 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.296268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.297194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.297641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.298073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] PATH_INFO: `/v3/projects/d0164a8160f8461988819f82476c4b8d` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.299403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0164a8160f8461988819f82476c4b8d)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.302072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.340734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fee73-f811-4505-a294-abda15ebe7ee None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.344288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.344925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.345202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.345529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.354640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.356047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.366136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 874/1746] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:10.394159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:10.412726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0164a8160f8461988819f82476c4b8d', 'request_id': 'req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:10.472358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.473788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.484272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0164a8160f8461988819f82476c4b8d', 'request_id': 'req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:10.488240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389ceb0-8e15-4a46-986b-57c560213926 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.491584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0164a8160f8461988819f82476c4b8d', 'request_id': 'req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:10.503523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] The token cache is being invalidate because project d0164a8160f8461988819f82476c4b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:10.504403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:10.505356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5f3b8dbc-0386-4e25-a1e1-09da7c4f3a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:10.507349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 873/1747] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:10 2022] DELETE /identity/v3/projects/d0164a8160f8461988819f82476c4b8d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:10.512280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.512592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.516944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.542205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.542875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.543155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.543659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.545813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.548441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:10.550139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.617356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.618740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d22f0-0a4b-4eb6-b6e6-2b2f82a0e810 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.654958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 875/1748] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:10.657530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 874/1749] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:10.666666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.667182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.667714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.707752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:10.707752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:10.711496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 876/1750] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:10.715521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.727614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd2e51-5244-4bf1-b8e1-5502a6f601a0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.736335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.737274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.737566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.737847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.738705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:10.739175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.741031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.757160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 875/1751] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:10.761272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.771699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe7135-6395-43d7-8d6f-e042f4c63d9a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:10.777949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:10.778696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:10.779644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:10.780000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:10.780757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:10.785730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:10.951586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 877/1752] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:10 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:10.955823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:10.966174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c12c65-dae7-4eda-bcb0-36a30fef1111 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:11.024085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:11.024722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.025003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.025278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.025945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:11.027848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:11.156789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 876/1753] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:10 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:11.157994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:11.170117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67da62a-ce11-48da-a76c-2de443d3217f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:11.175203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:11.176745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.177085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.177475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.178134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:11.179720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:11.196301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 878/1754] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:11.197804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:11.210501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5679ad-fca1-4aa7-86f0-c590c51da21e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:11.215445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b408308-f221-448f-92ed-1e8f97891cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:11.220297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b408308-f221-448f-92ed-1e8f97891cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.220297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b408308-f221-448f-92ed-1e8f97891cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.220297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b408308-f221-448f-92ed-1e8f97891cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.220297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b408308-f221-448f-92ed-1e8f97891cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:11.220297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b408308-f221-448f-92ed-1e8f97891cb2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:11.233245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 877/1755] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:11.237583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:11.249994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748137-fe17-431b-80db-b44be72f0487 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:11.257166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:11.258541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.258880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.259160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] PATH_INFO: `/v3/projects/678ed059651f49a08ad383ffcbbae3df/users/cb175ae8888f445cb253b5dad312ad21/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.274929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] RBAC: Authorizing `identity:create_grant(project_id=678ed059651f49a08ad383ffcbbae3df, user_id=cb175ae8888f445cb253b5dad312ad21, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:11.278722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:11.380951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:11.396223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 879/1756] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:11 2022] PUT /identity/v3/projects/678ed059651f49a08ad383ffcbbae3df/users/cb175ae8888f445cb253b5dad312ad21/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:11.421328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.421785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.421879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.467089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] MFA Rules not processed for user `cb175ae8888f445cb253b5dad312ad21`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:11.468458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:11.538432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 878/1757] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:11.542967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:11.603148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1124ff-7ca4-40f5-999e-df4156a9930e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:11.607711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:11.608383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.608669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.608942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.609585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:11.611440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:11.773198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 880/1758] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:11 2022] POST /identity/v3/projects => generated 367 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:11.774178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:11.786686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad98bf-75d8-4f0b-9f77-25c22118fed9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:11.850950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:11.851566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:11.851843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:11.852113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:11.852817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:11.854758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:12.039408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 879/1759] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:11 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:12.043666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:12.055022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a940-44e0-441c-a086-a5ed70342c22 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:12.059675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:12.060282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:12.060584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:12.060860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:12.061504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:12.063067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:12.083495 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 881/1760] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:12.087673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:12.097726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cec19a-bbc3-4f4d-aae5-1964c629620b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:12.102339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:12.102990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:12.103273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:12.103548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] PATH_INFO: `/v3/projects/a3a72801172943b3bdbe2a6e3236322c/users/fad68c00d08d4ce38f2ab4b519342778/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:12.107745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:12.119699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a72801172943b3bdbe2a6e3236322c, user_id=fad68c00d08d4ce38f2ab4b519342778, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:12.122370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:12.199281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 880/1761] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:12 2022] PUT /identity/v3/projects/a3a72801172943b3bdbe2a6e3236322c/users/fad68c00d08d4ce38f2ab4b519342778/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:12.203155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:12.213666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da861e0e-ba90-488d-820d-b11fe9d90e06 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:12.273115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afd8b0-32e3-40d7-8815-f13fe76fa57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:12.273879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afd8b0-32e3-40d7-8815-f13fe76fa57e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:12.274100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afd8b0-32e3-40d7-8815-f13fe76fa57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:12.274385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afd8b0-32e3-40d7-8815-f13fe76fa57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:12.275131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afd8b0-32e3-40d7-8815-f13fe76fa57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:12.276649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afd8b0-32e3-40d7-8815-f13fe76fa57e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:12.292851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 882/1762] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:12.295675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:12.306918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c7d5c-47c1-4cc4-aa87-0699797dc553 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:12.311441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:12.312133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:12.312450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:12.314686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] PATH_INFO: `/v3/projects/a3a72801172943b3bdbe2a6e3236322c/users/fad68c00d08d4ce38f2ab4b519342778/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:12.314686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a72801172943b3bdbe2a6e3236322c, user_id=fad68c00d08d4ce38f2ab4b519342778, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:12.318410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:12.328701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 881/1763] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:12 2022] PUT /identity/v3/projects/a3a72801172943b3bdbe2a6e3236322c/users/fad68c00d08d4ce38f2ab4b519342778/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:12.338221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:12.340041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86de98d-1085-45f9-b0f4-bbd768ca879b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:12.340340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86de98d-1085-45f9-b0f4-bbd768ca879b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:12.340634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86de98d-1085-45f9-b0f4-bbd768ca879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:12.368217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:12.369342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c86de98d-1085-45f9-b0f4-bbd768ca879b None None] MFA Rules not processed for user `fad68c00d08d4ce38f2ab4b519342778`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:12.370522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c86de98d-1085-45f9-b0f4-bbd768ca879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:12.442899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 883/1764] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:12.453679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:12.455466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:12.553152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f0f70-52db-435a-8adb-2fed1fe20476 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:12.557581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:12.558270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:12.558562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:12.558836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:12.571146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:12.573419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:12.596464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 882/1765] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:12.613981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:12.725340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:13.108706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0d2c7-2b35-4e07-a1ea-14acff577f40 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:13.109191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0d2c7-2b35-4e07-a1ea-14acff577f40 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:13.109294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0d2c7-2b35-4e07-a1ea-14acff577f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:13.194165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e7c0d2c7-2b35-4e07-a1ea-14acff577f40 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:13.195393 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e7c0d2c7-2b35-4e07-a1ea-14acff577f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:13.274194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 884/1766] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:04:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:13.347678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:13.680120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:13.681566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:13.824943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9349393-6f74-4f33-a3ba-132646d42ee7 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:13.830157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:13.830775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:13.831049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:13.831359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:13.840026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:13.841679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:13.855015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 883/1767] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:04:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:04:14.548966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:14.549463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:14.549852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:14.604073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:14.607228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:14.738640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 885/1768] 10.222.0.74 () {58 vars in 1015 bytes} [Thu Sep 1 18:04:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:14.785868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:14.797781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:14.837596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2ddc8-a380-4359-8cea-b323b055a0b8 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:14.871177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:14.871177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:14.871177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:14.871177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:14.883125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:14.887686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:14.919237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 884/1769] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:14.981876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:15.579386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:15.581183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:15.745305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:15.746741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:15.797922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1c47a-8f65-4936-923c-134310ef90d8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:15.805700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:15.807712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:15.808750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:15.809998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] PATH_INFO: `/v3/users/ef37438d7e0945babf1ec23bb9fd9ccf` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:15.811575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef37438d7e0945babf1ec23bb9fd9ccf)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:15.814098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:15.831960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:15.909337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2a58a-412c-4de0-a6fe-19ebe58fd681 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:15.916525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:15.917368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:15.917966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:15.918389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] PATH_INFO: `/v3/users/6f40b97920434d1492030ac0f4d89ffd` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:15.925245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f40b97920434d1492030ac0f4d89ffd)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:15.925245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:16.261743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef37438d7e0945babf1ec23bb9fd9ccf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.272251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef37438d7e0945babf1ec23bb9fd9ccf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.274709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef37438d7e0945babf1ec23bb9fd9ccf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.275494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef37438d7e0945babf1ec23bb9fd9ccf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.378991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f68e1357-5af3-422e-8456-b8ca1ed50ccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef37438d7e0945babf1ec23bb9fd9ccf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.388277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f40b97920434d1492030ac0f4d89ffd'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.413101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 886/1770] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:15 2022] DELETE /identity/v3/users/ef37438d7e0945babf1ec23bb9fd9ccf => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:16.418238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c48583-7f80-4762-8c5f-c74911344bc4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:16.422550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c48583-7f80-4762-8c5f-c74911344bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:16.422550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c48583-7f80-4762-8c5f-c74911344bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:16.426653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f40b97920434d1492030ac0f4d89ffd'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.461407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f0c48583-7f80-4762-8c5f-c74911344bc4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:16.464421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f0c48583-7f80-4762-8c5f-c74911344bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:16.522277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f40b97920434d1492030ac0f4d89ffd'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.523902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f40b97920434d1492030ac0f4d89ffd'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.677775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 887/1771] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:16.679826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f40b97920434d1492030ac0f4d89ffd'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:16.679826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 885/1772] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:15 2022] DELETE /identity/v3/users/6f40b97920434d1492030ac0f4d89ffd => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:16.684449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:16.684978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:16.688149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:16.699924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:16.702022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:16.774849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:16.776572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:16.851648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea40ee-370f-4ea4-84d6-4101f992a432 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:16.862233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1ac6-18f0-4409-9294-6504a30826ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:16.863426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1ac6-18f0-4409-9294-6504a30826ee service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:16.863862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1ac6-18f0-4409-9294-6504a30826ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:16.864582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1ac6-18f0-4409-9294-6504a30826ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:16.927798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:16.964363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 888/1773] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:17.016795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1ac6-18f0-4409-9294-6504a30826ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:17.018639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1ac6-18f0-4409-9294-6504a30826ee service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:17.037435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:17.047735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:17.052331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 886/1774] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:17.068750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44cd8c-c032-49df-949f-99d5e174b40a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:17.079723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4873523-aa0b-4a87-8f99-182acf039708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:17.080400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4873523-aa0b-4a87-8f99-182acf039708 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:17.082952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4873523-aa0b-4a87-8f99-182acf039708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:17.083240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4873523-aa0b-4a87-8f99-182acf039708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:17.106640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4873523-aa0b-4a87-8f99-182acf039708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:17.108112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4873523-aa0b-4a87-8f99-182acf039708 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:17.135494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 889/1775] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:18.341301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de391ff-1b0a-484e-9c09-03fee3b09776 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:18.342328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de391ff-1b0a-484e-9c09-03fee3b09776 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:18.343155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de391ff-1b0a-484e-9c09-03fee3b09776 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:18.345404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 887/1776] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:04:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:04:18.407769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:18.408278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:18.408413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:18.450880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:18.452887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:18.468084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:18.749537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 890/1777] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:18.892144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de391ff-1b0a-484e-9c09-03fee3b09776 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:18.897373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2de391ff-1b0a-484e-9c09-03fee3b09776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:18.948632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de391ff-1b0a-484e-9c09-03fee3b09776 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:18.969688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:18.971680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:18.972883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:18.973632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:18.997221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:19.000009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:19.020963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 888/1778] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:19.154015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:19.155133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:19.197723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409bd96-d761-4f6f-b067-18d5ee808ab5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:19.213479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:19.214295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:19.214810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:19.215163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] PATH_INFO: `/v3/projects/f7d9509296034fda840fc3b3548bfb5c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:19.218113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d9509296034fda840fc3b3548bfb5c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:19.220332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:19.244359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:19.251461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:19.285160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9271a25-d658-4b12-bac3-0037de82ac3a service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:19.297915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:19.303212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:19.303717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:19.304333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] PATH_INFO: `/v3/projects/34d77d812f2a422cbc65b8c6e633bd98` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:19.306971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d77d812f2a422cbc65b8c6e633bd98)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:19.310795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:19.435119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d9509296034fda840fc3b3548bfb5c', 'request_id': 'req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:19.749617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d77d812f2a422cbc65b8c6e633bd98', 'request_id': 'req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:19.906843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d9509296034fda840fc3b3548bfb5c', 'request_id': 'req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.007529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d9509296034fda840fc3b3548bfb5c', 'request_id': 'req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.007529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] The token cache is being invalidate because project f7d9509296034fda840fc3b3548bfb5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:20.007529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.007529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ef1ac4a9-0856-4fc9-b28b-191bf42d46d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.007529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 891/1779] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:19 2022] DELETE /identity/v3/projects/f7d9509296034fda840fc3b3548bfb5c => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:20.009193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2466c5-4eea-49af-996f-cfd451ea39f9 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:20.009193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2466c5-4eea-49af-996f-cfd451ea39f9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:20.009193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2466c5-4eea-49af-996f-cfd451ea39f9 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:20.009193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 892/1780] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:04:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:04:20.009193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9fef-b573-43cf-b4fb-7b8b9d4e53b0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:20.009193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9fef-b573-43cf-b4fb-7b8b9d4e53b0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:20.009925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9fef-b573-43cf-b4fb-7b8b9d4e53b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:20.009925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-817f9fef-b573-43cf-b4fb-7b8b9d4e53b0 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:20.009925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-817f9fef-b573-43cf-b4fb-7b8b9d4e53b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:20.127800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 893/1781] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:04:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:20.129616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:20.130188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:20.130761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:20.176911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:20.176911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:20.199610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d77d812f2a422cbc65b8c6e633bd98', 'request_id': 'req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.206559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d77d812f2a422cbc65b8c6e633bd98', 'request_id': 'req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.222063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] The token cache is being invalidate because project 34d77d812f2a422cbc65b8c6e633bd98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:20.223122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.224456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:20.227334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 889/1782] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:19 2022] DELETE /identity/v3/projects/34d77d812f2a422cbc65b8c6e633bd98 => generated 0 bytes in 985 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:20.228467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:20.230158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:20.306080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 894/1783] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:20.307362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:20.309077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:20.356145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f5f50-6a67-4d71-8fb2-ae4ae2f5c309 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:20.356145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:20.356145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:20.356145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:20.356559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:20.356559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:20.356715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ad1cb-6b06-49e8-a6e1-aabf45a2d957 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:20.368631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:20.369395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:20.369717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:20.370120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] PATH_INFO: `/v3/users/3781b1652ea84bb6bf8ca1f6bf02b1bb` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:20.372190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3781b1652ea84bb6bf8ca1f6bf02b1bb)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:20.374486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:20.398851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:20.400874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:20.417114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 895/1784] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:20.418456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:20.420245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:20.449178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0153b-c4c1-4d69-b778-dfa4d36025e8 service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:20.456344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:20.457873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:20.458287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:20.458806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] PATH_INFO: `/v3/users/e4ec01feb76542b8b9812ce818327ecf` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:20.461182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ec01feb76542b8b9812ce818327ecf)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:20.464082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:21.094940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ec01feb76542b8b9812ce818327ecf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.095880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3781b1652ea84bb6bf8ca1f6bf02b1bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.105104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ec01feb76542b8b9812ce818327ecf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.106515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ec01feb76542b8b9812ce818327ecf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.107443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ec01feb76542b8b9812ce818327ecf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.111145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3781b1652ea84bb6bf8ca1f6bf02b1bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.327585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e3ad1f23-6a47-441a-9ae7-0e7ab05ed3f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ec01feb76542b8b9812ce818327ecf'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.330019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:21.371765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 896/1785] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:20 2022] DELETE /identity/v3/users/e4ec01feb76542b8b9812ce818327ecf => generated 0 bytes in 953 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:21.373533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:21.373984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:21.374590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:21.410917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3781b1652ea84bb6bf8ca1f6bf02b1bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.413142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3781b1652ea84bb6bf8ca1f6bf02b1bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.455846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2fabc009-5ff1-4557-bb38-cb93ebcad2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3781b1652ea84bb6bf8ca1f6bf02b1bb'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:21.480658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 890/1786] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:20 2022] DELETE /identity/v3/users/3781b1652ea84bb6bf8ca1f6bf02b1bb => generated 0 bytes in 1252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:21.484357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:21.485011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:21.485307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:21.485584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:21.486861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:21.532635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:21.599397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:21.600838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 897/1787] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:21.601380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:21.615092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 891/1788] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:21.625029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:21.625277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd64f44-e529-4938-8846-4bd03b53a3b0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:21.626436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:21.643443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:21.652709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:21.653173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:21.653562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:21.654434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:21.654847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:21.657035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:21.678813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 898/1789] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:21.682917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:21.699461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881c644-012b-4fd9-ad37-27d6236de974 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:21.717918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:21.718837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:21.719275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:21.719706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:21.720699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:21.723395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:21.728570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f24d2-3aae-48d6-9461-3456a441fe99 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:21.734916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:21.735791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:21.736231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:21.736539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:21.749784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:21.751564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:21.770210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 892/1790] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:21.940850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 899/1791] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:21 2022] POST /identity/v3/projects => generated 355 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:21.943814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:21.943814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c0638-59d7-4bde-904b-fb2927ab2b5a service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:21.949574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:21.950345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:21.950635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:21.950915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:21.951583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:21.953580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:22.330938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 893/1792] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:21 2022] POST /identity/v3/users => generated 278 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:22.335068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:22.346595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e42e-daeb-41f0-88fa-fb87f7642375 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:22.353663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:22.354674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:22.354961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:22.355237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:22.360434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:22.360434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:22.372031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:22.373844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:22.394521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 900/1793] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:22.399826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec02ac-604d-4bcb-a893-16307edec0be admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:22.402873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:22.419773 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:22.422510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:22.422976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:22.423416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] PATH_INFO: `/v3/users/cb175ae8888f445cb253b5dad312ad21` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:22.425117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb175ae8888f445cb253b5dad312ad21)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:22.428522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838fcd6-16e5-46d5-8bf8-1841b59b659d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:22.428964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:22.437343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:22.438005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:22.438291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:22.438569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:22.439259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:22.441017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:22.458759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 901/1794] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:22.460335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:22.462145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:22.477921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c66b65-649c-4262-83eb-965b6746bd3a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:22.484587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:22.486309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:22.486831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:22.487261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] PATH_INFO: `/v3/projects/94db85b0d5bd453f8c1d3c2afb35ec56` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:22.491835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94db85b0d5bd453f8c1d3c2afb35ec56)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:22.491835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:22.995418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb175ae8888f445cb253b5dad312ad21'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.010074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94db85b0d5bd453f8c1d3c2afb35ec56', 'request_id': 'req-9c185a25-51ea-4f66-adb6-807ae3a37ec4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.024003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb175ae8888f445cb253b5dad312ad21'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.417424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94db85b0d5bd453f8c1d3c2afb35ec56', 'request_id': 'req-9c185a25-51ea-4f66-adb6-807ae3a37ec4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.425573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94db85b0d5bd453f8c1d3c2afb35ec56', 'request_id': 'req-9c185a25-51ea-4f66-adb6-807ae3a37ec4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.447286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] The token cache is being invalidate because project 94db85b0d5bd453f8c1d3c2afb35ec56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:23.447780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.450915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.453237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 902/1795] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:22 2022] DELETE /identity/v3/projects/94db85b0d5bd453f8c1d3c2afb35ec56 => generated 0 bytes in 994 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:23.458655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:23.458655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:23.480813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb175ae8888f445cb253b5dad312ad21'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.481622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb175ae8888f445cb253b5dad312ad21'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.508178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb175ae8888f445cb253b5dad312ad21'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.518608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 894/1796] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:22 2022] DELETE /identity/v3/users/cb175ae8888f445cb253b5dad312ad21 => generated 0 bytes in 1147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:23.520743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:23.540030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:23.558498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c185a25-51ea-4f66-adb6-807ae3a37ec4 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:23.627558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50150-d0b4-4400-b064-90fef8dee6d0 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:23.628460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a56ff1-a3cd-4be3-b026-cf7f3df62775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:23.629250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a56ff1-a3cd-4be3-b026-cf7f3df62775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:23.629602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a56ff1-a3cd-4be3-b026-cf7f3df62775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:23.629951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a56ff1-a3cd-4be3-b026-cf7f3df62775 admin admin] PATH_INFO: `/v3/projects/8901804e235442578d0e2120fbb2f7ac/users/4449dcfa2ed244eb831cc2fbf7d7ef4f/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:23.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:23.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:23.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:23.636211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] PATH_INFO: `/v3/projects/6c7163d42a4f48a59e46564e7d8b0fb4` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:23.641217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7163d42a4f48a59e46564e7d8b0fb4)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:23.643491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:23.648009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a56ff1-a3cd-4be3-b026-cf7f3df62775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8901804e235442578d0e2120fbb2f7ac, user_id=4449dcfa2ed244eb831cc2fbf7d7ef4f, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:23.650981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a56ff1-a3cd-4be3-b026-cf7f3df62775 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:23.835961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 903/1797] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:23 2022] PUT /identity/v3/projects/8901804e235442578d0e2120fbb2f7ac/users/4449dcfa2ed244eb831cc2fbf7d7ef4f/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:23.835961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa83f2-1fac-4a49-8ebb-edf4e43d92dc None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:23.835961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa83f2-1fac-4a49-8ebb-edf4e43d92dc None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:23.835961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa83f2-1fac-4a49-8ebb-edf4e43d92dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:23.880692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7163d42a4f48a59e46564e7d8b0fb4', 'request_id': 'req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:23.884486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-17fa83f2-1fac-4a49-8ebb-edf4e43d92dc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:23.885806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-17fa83f2-1fac-4a49-8ebb-edf4e43d92dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.019656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 904/1798] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:24.019656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320c0f-50af-4bde-b997-fcf92910887f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.019656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320c0f-50af-4bde-b997-fcf92910887f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.019656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320c0f-50af-4bde-b997-fcf92910887f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.061807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e8320c0f-50af-4bde-b997-fcf92910887f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:24.063282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e8320c0f-50af-4bde-b997-fcf92910887f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.076331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 905/1799] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:24.081910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.103503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.103503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.124686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] MFA Rules not processed for user `4449dcfa2ed244eb831cc2fbf7d7ef4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:24.126881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7163d42a4f48a59e46564e7d8b0fb4', 'request_id': 'req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.127524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.137215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7163d42a4f48a59e46564e7d8b0fb4', 'request_id': 'req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.152019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] The token cache is being invalidate because project 6c7163d42a4f48a59e46564e7d8b0fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:24.152346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.153211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1e0b98ae-7d47-4a86-ae15-6ab4cf5abcfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.157989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 895/1800] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:23 2022] DELETE /identity/v3/projects/6c7163d42a4f48a59e46564e7d8b0fb4 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:24.157989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.157989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.157989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.195535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:24.196818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.205742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 896/1801] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:24.210580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.210580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.230272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 906/1802] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:24.231166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.232901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.260209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872adc0-7d4c-4cab-9a79-086753426d26 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:24.266709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:24.267369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.267654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.267937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] PATH_INFO: `/v3/users/178bea54a3aa4ff39f3d665f6a0149e1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.269228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178bea54a3aa4ff39f3d665f6a0149e1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:24.271653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:24.290429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e163332-7e75-4b09-8671-b7f3f874bd87 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:24.294211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:24.294912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.295265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.295607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.297516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.311508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:24.313168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:24.324326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 897/1803] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:24.325289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.334558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d067bc-136b-4be6-b97d-473a83b4a129 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:24.339326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:24.340125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.340463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.343502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.343502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:24.343502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:24.344492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:24.357099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 898/1804] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:24.358017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.365570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.380122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba60f16-5422-49e3-8a68-47250002205f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:24.385969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:24.386279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.386565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.386844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.387488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:24.389476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:24.570683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 899/1805] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/projects => generated 357 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:24.570683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.617988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144159d7-4eaf-438b-8dc9-73bad0606a82 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:24.622671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:24.623277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.623530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.623799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.624467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:24.626353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:24.741857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178bea54a3aa4ff39f3d665f6a0149e1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.749064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178bea54a3aa4ff39f3d665f6a0149e1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.749786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178bea54a3aa4ff39f3d665f6a0149e1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.750397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178bea54a3aa4ff39f3d665f6a0149e1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.799724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 900/1806] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/projects => generated 371 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:24.799724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.800825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:24.864380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eec08-8fe3-4b79-9398-4628ee180118 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:24.869071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:24.869680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:24.869966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:24.870244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:24.870879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:24.872843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:24.929727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178bea54a3aa4ff39f3d665f6a0149e1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:24.948031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 907/1807] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:24 2022] DELETE /identity/v3/users/178bea54a3aa4ff39f3d665f6a0149e1 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:24.948828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:24.950163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:25.016219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1dd97-a4ff-4cb5-9377-9184291a02e6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.021447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.023275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.023275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.023275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.023275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.025228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.061219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 901/1808] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:25.061219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.062989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:25.077653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d878-6da6-4387-89d2-79d0478efc4d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.086063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.086792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.087176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.087452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] PATH_INFO: `/v3/projects/678ed059651f49a08ad383ffcbbae3df` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.088692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=678ed059651f49a08ad383ffcbbae3df)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.090630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.224697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 908/1809] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:24 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:25.226303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.247718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccd435-c5e2-418b-895e-7b4a930b1d64 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.263016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678ed059651f49a08ad383ffcbbae3df', 'request_id': 'req-6d519b83-87b1-4b29-a966-1b8a8caaf122'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.321906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.323438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.323438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.323438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.323767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.325476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.339490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 909/1810] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:25.339490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.348067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c50e9a-69ea-4c2d-a549-cdccd5ad2137 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.358134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.358761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.359040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.359315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.360103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.361718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.379328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 910/1811] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:25.379328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.382812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678ed059651f49a08ad383ffcbbae3df', 'request_id': 'req-6d519b83-87b1-4b29-a966-1b8a8caaf122'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.387894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6bfe-5631-4632-95b7-5bb80051a441 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.389225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678ed059651f49a08ad383ffcbbae3df', 'request_id': 'req-6d519b83-87b1-4b29-a966-1b8a8caaf122'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.393751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.394120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.394557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.394999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] PATH_INFO: `/v3/projects/c9630a84df434d3aaa5b203e88d69eb5/users/be4aed92790e4f638f08aea9c1520b24/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.401942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] The token cache is being invalidate because project 678ed059651f49a08ad383ffcbbae3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:25.402629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.403210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.404855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 902/1812] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:25 2022] DELETE /identity/v3/projects/678ed059651f49a08ad383ffcbbae3df => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:25.408109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.408109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:25.416170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9630a84df434d3aaa5b203e88d69eb5, user_id=be4aed92790e4f638f08aea9c1520b24, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.418886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.420620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519b83-87b1-4b29-a966-1b8a8caaf122 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.425852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.426470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.426738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.427004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.427651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.429266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.445108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 903/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:25.445108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.445108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:25.454007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a9f88-4ae9-4a7a-a1e6-dc334c92278b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.458517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.459117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.459394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.459665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] PATH_INFO: `/v3/users/fad68c00d08d4ce38f2ab4b519342778` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.460810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad68c00d08d4ce38f2ab4b519342778)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.462673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.642232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 911/1814] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:25 2022] PUT /identity/v3/projects/c9630a84df434d3aaa5b203e88d69eb5/users/be4aed92790e4f638f08aea9c1520b24/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:25.652969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.708645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ab73-115f-46c0-952a-d0ab0c60333c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:25.714299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:25.727530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:25.727530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:25.727530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] PATH_INFO: `/v3/projects/76d27266cf0d45a9b830b5fcd107ea5f/users/ed407052035d4863a29bc2254d38195d/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:25.731832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d27266cf0d45a9b830b5fcd107ea5f, user_id=ed407052035d4863a29bc2254d38195d, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:25.734546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:25.832540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad68c00d08d4ce38f2ab4b519342778'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.839562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad68c00d08d4ce38f2ab4b519342778'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.981892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad68c00d08d4ce38f2ab4b519342778'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.982586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad68c00d08d4ce38f2ab4b519342778'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:25.988170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 912/1815] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:25 2022] PUT /identity/v3/projects/76d27266cf0d45a9b830b5fcd107ea5f/users/ed407052035d4863a29bc2254d38195d/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:25.989118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:25.990764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.005264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad68c00d08d4ce38f2ab4b519342778'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:26.011510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 904/1816] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:25 2022] DELETE /identity/v3/users/fad68c00d08d4ce38f2ab4b519342778 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:26.012360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.013683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.059223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6627f25-98e7-4e76-b085-4f5ea70cb8eb admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.112870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6954d3-923e-4820-9c74-d53eb1909b9d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.124330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05922c-1cd6-47d9-9ab3-37b966a12b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.124330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05922c-1cd6-47d9-9ab3-37b966a12b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.124330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05922c-1cd6-47d9-9ab3-37b966a12b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.124330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05922c-1cd6-47d9-9ab3-37b966a12b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.124715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05922c-1cd6-47d9-9ab3-37b966a12b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.124715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05922c-1cd6-47d9-9ab3-37b966a12b02 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.126239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.126896 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.127175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.127447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] PATH_INFO: `/v3/projects/239f48cf6aea468eaf63125493a56cdf` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.130312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=239f48cf6aea468eaf63125493a56cdf)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.132213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.136775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 905/1817] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:26.138590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.138590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.138865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.168546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] MFA Rules not processed for user `ed407052035d4863a29bc2254d38195d`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:26.180640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.235962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 906/1818] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:26.236746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.248697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da41555-91f6-4946-ab93-00201da5b31f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.253625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.254442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.254816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.255189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] PATH_INFO: `/v3/projects/c9630a84df434d3aaa5b203e88d69eb5/users/be4aed92790e4f638f08aea9c1520b24/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.257268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9630a84df434d3aaa5b203e88d69eb5, user_id=be4aed92790e4f638f08aea9c1520b24, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.259880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.279204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 907/1819] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:26 2022] PUT /identity/v3/projects/c9630a84df434d3aaa5b203e88d69eb5/users/be4aed92790e4f638f08aea9c1520b24/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:26.279204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.279204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.314445 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239f48cf6aea468eaf63125493a56cdf', 'request_id': 'req-11eb5dd4-09d2-459d-9d07-0fd205492d6a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:26.361054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39935b94-0c0d-44a3-b503-81ab2230ca8e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.437573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.438835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.439331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.439728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.440484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.443179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.452370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239f48cf6aea468eaf63125493a56cdf', 'request_id': 'req-11eb5dd4-09d2-459d-9d07-0fd205492d6a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:26.460631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239f48cf6aea468eaf63125493a56cdf', 'request_id': 'req-11eb5dd4-09d2-459d-9d07-0fd205492d6a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:26.478822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] The token cache is being invalidate because project 239f48cf6aea468eaf63125493a56cdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:26.479117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:26.479884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-11eb5dd4-09d2-459d-9d07-0fd205492d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:26.484838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 913/1820] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:25 2022] DELETE /identity/v3/projects/239f48cf6aea468eaf63125493a56cdf => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:26.484838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8856-8b2e-4694-a56a-234c65f6e601 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.484838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8856-8b2e-4694-a56a-234c65f6e601 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.484838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8856-8b2e-4694-a56a-234c65f6e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.513977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-dc0b8856-8b2e-4694-a56a-234c65f6e601 None None] MFA Rules not processed for user `be4aed92790e4f638f08aea9c1520b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:26.515196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dc0b8856-8b2e-4694-a56a-234c65f6e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.584223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 914/1821] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:26.585512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.585797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.586069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.598560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 908/1822] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:26 2022] POST /identity/v3/projects => generated 375 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:26.641859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.641859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.642304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:26.642304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.658410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fbeaa-0504-4884-a7c5-545b397da3ad admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.664165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.664833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.665114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.665386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.675356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 915/1823] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:26.677153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.683230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.697062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742a2c6-94e2-414c-b696-fa1a7819be91 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.705408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e9d65-5e91-4671-a18d-13c548c52224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.706066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e9d65-5e91-4671-a18d-13c548c52224 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.706402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e9d65-5e91-4671-a18d-13c548c52224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.707502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e9d65-5e91-4671-a18d-13c548c52224 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.708435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e9d65-5e91-4671-a18d-13c548c52224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.710523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e9d65-5e91-4671-a18d-13c548c52224 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.737264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.739249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.749460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 909/1824] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:26.750369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.751774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.769602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6271f1-4e5a-4c4e-9a67-2f6cb1bdfdbe service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.774803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.775458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.775732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.776010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] PATH_INFO: `/v3/projects/a3a72801172943b3bdbe2a6e3236322c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.777188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3a72801172943b3bdbe2a6e3236322c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.779089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.822372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 916/1825] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:26 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:26.823832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.824115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.824553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.844233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:26.845462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:26.853479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 917/1826] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:26.854265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.863742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaa7f2-ff6c-43d0-b8b4-60bce6729960 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.868481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.869093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.869373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.869645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.870314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.871958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.882965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 918/1827] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:26.883767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.893317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a115ff-76e7-4f85-841c-45c2c4dd6635 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.899620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.899620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.899620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.900113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.900347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:26.900656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.902243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:26.915207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 919/1828] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:26.916013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:26.925411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47c5f8-8d6b-4710-b337-ce3be7de0505 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:26.930005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:26.930651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:26.930929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:26.931204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] PATH_INFO: `/v3/projects/fff6ccefd04b4955b0b14fc817fde974/users/65b7c5df908545fd8a5d64593b330707/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:26.947175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff6ccefd04b4955b0b14fc817fde974, user_id=65b7c5df908545fd8a5d64593b330707, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:26.949942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.083440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3a72801172943b3bdbe2a6e3236322c', 'request_id': 'req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:27.084566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 920/1829] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:26 2022] PUT /identity/v3/projects/fff6ccefd04b4955b0b14fc817fde974/users/65b7c5df908545fd8a5d64593b330707/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:27.088206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.150723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1fe3f-9960-4009-9502-f1c0d3ae0210 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.210454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e25b-776e-4aaf-b2a5-8318a5f8e1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.210960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e25b-776e-4aaf-b2a5-8318a5f8e1ca admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.211238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e25b-776e-4aaf-b2a5-8318a5f8e1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.211510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e25b-776e-4aaf-b2a5-8318a5f8e1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.212150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39e25b-776e-4aaf-b2a5-8318a5f8e1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.214051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39e25b-776e-4aaf-b2a5-8318a5f8e1ca admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.255926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3a72801172943b3bdbe2a6e3236322c', 'request_id': 'req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:27.257086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:27.262746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3a72801172943b3bdbe2a6e3236322c', 'request_id': 'req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:27.274867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] The token cache is being invalidate because project a3a72801172943b3bdbe2a6e3236322c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:27.275245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:27.275978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:27.277625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 910/1830] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:26 2022] DELETE /identity/v3/projects/a3a72801172943b3bdbe2a6e3236322c => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:27.280920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.280920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.293205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b35c-42ee-4e71-a3f6-03d74d6afcd6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.298326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.308903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.308903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.308903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.308903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.308903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.312823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 911/1831] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:27.316852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.316852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.372166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 921/1832] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:27 2022] POST /identity/v3/projects => generated 387 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:27.372166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.372166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.372166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.374217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212b394-43b3-45ea-9af5-f3e1c931ad3b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.395499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:27.396757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.425826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.426941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.427836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.428117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.430077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.465757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 922/1833] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:27.465757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.472734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce79ff5-74de-4159-9da3-47ab393d36a3 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.478481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.479226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.479571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.479928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] PATH_INFO: `/v3/projects/fff6ccefd04b4955b0b14fc817fde974/users/65b7c5df908545fd8a5d64593b330707/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.482104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff6ccefd04b4955b0b14fc817fde974, user_id=65b7c5df908545fd8a5d64593b330707, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.485167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.497436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 923/1834] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:27 2022] PUT /identity/v3/projects/fff6ccefd04b4955b0b14fc817fde974/users/65b7c5df908545fd8a5d64593b330707/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:27.498339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.499876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.501761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cbf86-f8b0-4b02-bf3a-40848fab613b service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.506824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:27.511279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.513943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 912/1835] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:27.515383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c697-87f0-406b-9f50-195e6beeebfc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.515847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c697-87f0-406b-9f50-195e6beeebfc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.516267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c697-87f0-406b-9f50-195e6beeebfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.570494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7a78c697-87f0-406b-9f50-195e6beeebfc None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:27.571692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7a78c697-87f0-406b-9f50-195e6beeebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.658800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ce273-2389-462d-af81-6554d7624f47 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.666607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.667451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.668048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.668519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.669326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.671649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.684747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 913/1836] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:27.689913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.689913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.689913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.733098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] MFA Rules not processed for user `65b7c5df908545fd8a5d64593b330707`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:27.734678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.805977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 924/1837] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:27 2022] POST /identity/v3/users => generated 294 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:27.808466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.830048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1eac2-a104-44bd-938c-106fdfed5aef admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.833658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 914/1838] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:27.838614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.839250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.839529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.839851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.841277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:27.841583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.843433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.847810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.860014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 925/1839] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:27.860814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711e448-50f3-4adb-a02d-92b3356bfe3b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.861994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.865141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:27.880287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.881140 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.881537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.883288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.884111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.888104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.919564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 915/1840] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:27.921043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:27.933923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a868-7461-4c3e-9fd2-f683082ada75 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.940342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.942372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.942764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.943136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.943901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:27.944697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2accdb-c943-4c35-9442-0a2c5a20b728 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:27.946163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:27.948386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:27.949205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:27.949541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:27.949905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:27.952079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:28.031778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.033519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.043686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 926/1841] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:28.044678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.074121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52537a0-5a43-4a13-be02-b86706641571 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.086263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.100471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 916/1842] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:27 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:28.101483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.102022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.102520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.105007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.113959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.120956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.145304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 927/1843] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:28.149542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.172119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeec623-c900-4b3d-93e4-65e892eabe0b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.274452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f97b7-e424-4998-9ad1-63b42ba40e27 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.282676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.286593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.287034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.287850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.288694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.291127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.293890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f55fc-b507-491f-a008-381ae219f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.306877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f55fc-b507-491f-a008-381ae219f5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.306877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f55fc-b507-491f-a008-381ae219f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.306877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f55fc-b507-491f-a008-381ae219f5b7 admin admin] PATH_INFO: `/v3/projects/17bc6846fc074abaab3c0a198ea8d026/users/64ef13d7a9e2419db08485d54ec61c76/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.330293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f55fc-b507-491f-a008-381ae219f5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bc6846fc074abaab3c0a198ea8d026, user_id=64ef13d7a9e2419db08485d54ec61c76, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.334517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f55fc-b507-491f-a008-381ae219f5b7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.404981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 917/1844] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:28 2022] POST /identity/v3/users => generated 296 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:28.406843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.417866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ded64-21e0-4635-99d6-6598cf047a5b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.422884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.423533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.423848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.424122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.424819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.433698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.437756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 928/1845] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:28 2022] PUT /identity/v3/projects/17bc6846fc074abaab3c0a198ea8d026/users/64ef13d7a9e2419db08485d54ec61c76/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:28.449792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 918/1846] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:28.450927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.451044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.451340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.451619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.462966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58d63d-7a33-4c91-a0bd-4cf36ee4efda admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.492219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] MFA Rules not processed for user `64ef13d7a9e2419db08485d54ec61c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:28.493776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:28.528640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.529293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.529577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.529844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.530517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.532408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.546510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 919/1847] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:28.547365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.557722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be222fa0-a538-4ce1-b026-ffe1d8e0b890 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.562874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d0171-f0e3-4e82-aa14-c80acc7c1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.563531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d0171-f0e3-4e82-aa14-c80acc7c1b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.563809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d0171-f0e3-4e82-aa14-c80acc7c1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.564107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d0171-f0e3-4e82-aa14-c80acc7c1b05 admin admin] PATH_INFO: `/v3/projects/1dac52ef2df542a18fee84878e22ffc6/users/027c865899e74daa90952968bd230f68/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.568460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 929/1848] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:28.572908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.580222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d0171-f0e3-4e82-aa14-c80acc7c1b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dac52ef2df542a18fee84878e22ffc6, user_id=027c865899e74daa90952968bd230f68, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.583715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d0171-f0e3-4e82-aa14-c80acc7c1b05 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.584333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd342ef0-3e89-4a2e-8554-184b1f5136a0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.589139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.589745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.590055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.590331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.591071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.593074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.718963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 920/1849] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:28 2022] PUT /identity/v3/projects/1dac52ef2df542a18fee84878e22ffc6/users/027c865899e74daa90952968bd230f68/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:28.731270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.731562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.731843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.747251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 930/1850] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:28 2022] POST /identity/v3/projects => generated 389 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:28.750918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.790171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] MFA Rules not processed for user `027c865899e74daa90952968bd230f68`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:28.791397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:28.852298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcd12a-c8e0-4f53-8b85-26072a88d65a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.857569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.858429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.858813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.859182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.859947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.860456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 921/1851] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:28.862246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.865393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:28.876766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a746327-375c-452b-a42a-fa8efaaba0b8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:28.882272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:28.882962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:28.883303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:28.883578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:28.884212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:28.886159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:28.999308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 931/1852] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:28 2022] POST /identity/v3/users => generated 301 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:29.000434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.020556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 922/1853] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:28 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:29.024808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3c14f-4c81-4ee2-a3ca-1f4b6f788f24 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.025221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.101514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.102295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.102853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.103252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.103998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.105789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.116261 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5436f-e731-47d3-8a38-6e4e06c92aea admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.121324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 932/1854] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:29.121634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.122304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.122641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.122969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.123658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.125403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.125847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.141590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edfaea-8439-4119-bd21-7d110614888c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.146539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.147224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.147505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.147782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] PATH_INFO: `/v3/projects/c58f14df14944f21a13a9f1423df649f/users/257c9ae77731443cba9cbf5913ee7a20/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.163117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58f14df14944f21a13a9f1423df649f, user_id=257c9ae77731443cba9cbf5913ee7a20, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.165786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.319333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 933/1855] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:29 2022] PUT /identity/v3/projects/c58f14df14944f21a13a9f1423df649f/users/257c9ae77731443cba9cbf5913ee7a20/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:29.320627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.323496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 923/1856] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:29 2022] POST /identity/v3/users => generated 303 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:29.328519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.356260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92932f-492d-4017-bd80-7df4f9c7118f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.435808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8de2a-604a-4463-97ef-ea029a3fe841 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.446701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.447526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.450982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.450982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.450982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.451250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.451362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.451444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.451444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.451725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.452583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.454154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.465731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 924/1857] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:29.466918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.467911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 934/1858] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:29.471796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.481437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c6318-6196-4463-983e-617a5c51ecc2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.482871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6acfe-169a-497e-8a3f-a71b4d8f95f1 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.487316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.488396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddeed0d-0910-40f8-bd15-0b4b03eb440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.489063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.489352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.489470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeed0d-0910-40f8-bd15-0b4b03eb440e admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.489636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] PATH_INFO: `/v3/projects/305cb5c10f694c5683c8d79bbaf9c038/users/fedbdf68b9e741968d88f9c531c7a0d0/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.489889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeed0d-0910-40f8-bd15-0b4b03eb440e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.490344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeed0d-0910-40f8-bd15-0b4b03eb440e admin admin] PATH_INFO: `/v3/projects/c58f14df14944f21a13a9f1423df649f/users/257c9ae77731443cba9cbf5913ee7a20/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.497356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddeed0d-0910-40f8-bd15-0b4b03eb440e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58f14df14944f21a13a9f1423df649f, user_id=257c9ae77731443cba9cbf5913ee7a20, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.501602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddeed0d-0910-40f8-bd15-0b4b03eb440e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.510649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305cb5c10f694c5683c8d79bbaf9c038, user_id=fedbdf68b9e741968d88f9c531c7a0d0, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.513513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.519268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 935/1859] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:29 2022] PUT /identity/v3/projects/c58f14df14944f21a13a9f1423df649f/users/257c9ae77731443cba9cbf5913ee7a20/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:29.529931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.530218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.530487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.560250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] MFA Rules not processed for user `257c9ae77731443cba9cbf5913ee7a20`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:29.561780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:29.630095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 936/1860] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:29.640553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.658942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 925/1861] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:29 2022] PUT /identity/v3/projects/305cb5c10f694c5683c8d79bbaf9c038/users/fedbdf68b9e741968d88f9c531c7a0d0/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:29.665950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.695707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f555d-bc11-410c-b855-31f6b00d0560 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.731562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234cfe-7222-48b4-8ea6-a7a794cecb25 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.737399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.738149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.738542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.738826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.739490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.741329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.752063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.752063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.752063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.752063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.755141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 926/1862] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:29.758851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.770926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e900-4ad2-4005-9280-2415a2f5ac5b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.775898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141c604-49ae-4232-a084-f53f0c90abc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.776591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141c604-49ae-4232-a084-f53f0c90abc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.787452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141c604-49ae-4232-a084-f53f0c90abc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.787452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141c604-49ae-4232-a084-f53f0c90abc7 admin admin] PATH_INFO: `/v3/projects/305cb5c10f694c5683c8d79bbaf9c038/users/fedbdf68b9e741968d88f9c531c7a0d0/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.787452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141c604-49ae-4232-a084-f53f0c90abc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305cb5c10f694c5683c8d79bbaf9c038, user_id=fedbdf68b9e741968d88f9c531c7a0d0, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.787452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141c604-49ae-4232-a084-f53f0c90abc7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.793009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 927/1863] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:29 2022] PUT /identity/v3/projects/305cb5c10f694c5683c8d79bbaf9c038/users/fedbdf68b9e741968d88f9c531c7a0d0/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:29.803608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.803899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.804180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.819669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.827606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:29.831182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 937/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:29.834060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] MFA Rules not processed for user `fedbdf68b9e741968d88f9c531c7a0d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:29.835270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:29.910284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 928/1865] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:29.923882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.930590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac931c-4217-438f-a7d3-901c0968c76f service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:29.975501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:29.986109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:29.986745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:29.987014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:29.987287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:29.997180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:29.998600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:30.008462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 938/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:29 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:30.040532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cc66c-ba84-4acd-a242-22be5833b8b2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:30.045360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:30.045937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:30.046215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:30.046486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] PATH_INFO: `/v3/users/64ef13d7a9e2419db08485d54ec61c76` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:30.047758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ef13d7a9e2419db08485d54ec61c76)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:30.049799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:30.065322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:30.073761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc67e5-0586-4fee-9a56-9d3b3cef586e service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:30.077377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbf6b6-fd7e-4877-9acf-41aea247dfea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:30.077967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbf6b6-fd7e-4877-9acf-41aea247dfea service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:30.078243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbf6b6-fd7e-4877-9acf-41aea247dfea service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:30.078514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbf6b6-fd7e-4877-9acf-41aea247dfea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:30.148885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbf6b6-fd7e-4877-9acf-41aea247dfea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:30.148885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbf6b6-fd7e-4877-9acf-41aea247dfea service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:30.153277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 939/1867] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:30.316129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ef13d7a9e2419db08485d54ec61c76'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:30.323777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ef13d7a9e2419db08485d54ec61c76'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:30.435207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ef13d7a9e2419db08485d54ec61c76'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:30.435904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ef13d7a9e2419db08485d54ec61c76'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:30.452075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ef13d7a9e2419db08485d54ec61c76'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:30.461493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 929/1868] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:29 2022] DELETE /identity/v3/users/64ef13d7a9e2419db08485d54ec61c76 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:30.465404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:30.465698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:30.465977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:30.485671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:30.486889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:30.555566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 940/1869] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:30.567958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:30.573848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:30.651045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1122fbd-5755-4615-8dfa-25c61bbbd85b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:30.654870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:30.655571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:30.655935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:30.656598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:30.668390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:30.670351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:30.682667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 930/1870] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:31.298776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:31.303844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:31.331149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e59789-2ffc-412d-9207-1b8d9c6c9105 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:31.341106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:31.341761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:31.342138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:31.342907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] PATH_INFO: `/v3/projects/17bc6846fc074abaab3c0a198ea8d026` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:31.344716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bc6846fc074abaab3c0a198ea8d026)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:31.347320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:31.472724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:31.509641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bc6846fc074abaab3c0a198ea8d026', 'request_id': 'req-945cc715-eacc-42cb-92f0-7bea456a8376'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:31.619166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:31.672585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bc6846fc074abaab3c0a198ea8d026', 'request_id': 'req-945cc715-eacc-42cb-92f0-7bea456a8376'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:31.677182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bc6846fc074abaab3c0a198ea8d026', 'request_id': 'req-945cc715-eacc-42cb-92f0-7bea456a8376'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:31.688667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] The token cache is being invalidate because project 17bc6846fc074abaab3c0a198ea8d026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:31.688968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:31.689782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:31.692385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 941/1871] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:31 2022] DELETE /identity/v3/projects/17bc6846fc074abaab3c0a198ea8d026 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:31.696333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:31.707411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03b8ec-1adf-4145-947a-6dce4fddc6f7 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:31.784234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:31.784870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:31.785149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:31.785420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] PATH_INFO: `/v3/users/257c9ae77731443cba9cbf5913ee7a20` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:31.786713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] RBAC: Authorizing `identity:delete_user(user_id=257c9ae77731443cba9cbf5913ee7a20)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:31.788777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:32.077875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257c9ae77731443cba9cbf5913ee7a20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:32.085958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257c9ae77731443cba9cbf5913ee7a20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:32.195596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257c9ae77731443cba9cbf5913ee7a20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:32.196495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257c9ae77731443cba9cbf5913ee7a20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:32.233105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3be3b1d0-5df1-4c15-8eaf-1862ea0d677a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257c9ae77731443cba9cbf5913ee7a20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:32.245947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 931/1872] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:31 2022] DELETE /identity/v3/users/257c9ae77731443cba9cbf5913ee7a20 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:32.825121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:32.827020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:32.848305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cc715-eacc-42cb-92f0-7bea456a8376 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:32.915968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:32.916678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:32.916967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:32.917240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] PATH_INFO: `/v3/projects/c58f14df14944f21a13a9f1423df649f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:32.918423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58f14df14944f21a13a9f1423df649f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:32.920329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:33.087031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58f14df14944f21a13a9f1423df649f', 'request_id': 'req-a1284111-5e3c-43d0-9406-b5925dccc019'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:33.225193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58f14df14944f21a13a9f1423df649f', 'request_id': 'req-a1284111-5e3c-43d0-9406-b5925dccc019'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:33.231360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58f14df14944f21a13a9f1423df649f', 'request_id': 'req-a1284111-5e3c-43d0-9406-b5925dccc019'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:33.262355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] The token cache is being invalidate because project c58f14df14944f21a13a9f1423df649f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:33.262658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:33.263388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a1284111-5e3c-43d0-9406-b5925dccc019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:33.266557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 942/1873] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:32 2022] DELETE /identity/v3/projects/c58f14df14944f21a13a9f1423df649f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:33.270758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.271211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.271448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.301224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:33.301224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:33.358992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 932/1874] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:33.368381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.368668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.368945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.389572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:33.390912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:33.399875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 943/1875] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:33.404100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:33.414534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedf1c3-6471-41f4-8ec0-ffefb33fbd02 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:33.419359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:33.421305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.421305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.421305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.421305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:33.421641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:33.423109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:33.435128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 933/1876] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:33.439693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:33.450688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0c99e-b960-4d21-9314-ded11c816eff None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:33.455549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:33.456941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.457164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.457442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.458126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:33.460100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:33.570498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 944/1877] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:33 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:33.579924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:33.614660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7532ed-07aa-4a63-8479-1018f377aa25 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:33.688132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:33.689059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.689341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.689769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.690313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:33.693371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:33.797606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 934/1878] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:33 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:33.802376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:33.812874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc2f18-68c4-4e67-b78a-b0f9a9f47bb7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:33.817727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:33.818359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.818643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.818917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.819580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:33.821400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:33.834031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 945/1879] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:33.837556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:33.848758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808f375-732f-4d55-953b-507940e99175 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:33.853862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4e24a-4143-4774-b3f8-850e6a4de365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:33.854496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4e24a-4143-4774-b3f8-850e6a4de365 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.854781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4e24a-4143-4774-b3f8-850e6a4de365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.862863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4e24a-4143-4774-b3f8-850e6a4de365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.862863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4e24a-4143-4774-b3f8-850e6a4de365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:33.862863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4e24a-4143-4774-b3f8-850e6a4de365 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:33.872670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 935/1880] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:33.873990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:33.884218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5b33a-1bce-4944-bed3-9fc8e12153d7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:33.889195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:33.889877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:33.890162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:33.890439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] PATH_INFO: `/v3/projects/9c733dc7bd18452c9132b562e7481e42/users/623c092ff1314a66b11362582951abd6/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:33.909270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c733dc7bd18452c9132b562e7481e42, user_id=623c092ff1314a66b11362582951abd6, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:33.917158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:34.076511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 946/1881] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:33 2022] PUT /identity/v3/projects/9c733dc7bd18452c9132b562e7481e42/users/623c092ff1314a66b11362582951abd6/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:34.078038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:34.078548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:34.079016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:34.141409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] MFA Rules not processed for user `623c092ff1314a66b11362582951abd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:34.143051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:34.244176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:34.245035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 936/1882] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:34.336856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e27c0c-7cc0-45e5-92e0-d6032842e68c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:34.342243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:34.344784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:34.344784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:34.344784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:34.344784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:34.346273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:34.462637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 947/1883] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:34 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:34.466973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:34.478176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deb457-4f4d-4480-94f7-ccd907fa01be None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:34.535030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:34.535646 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:34.535924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:34.536204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:34.536897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:34.538859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:34.621819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:34.623584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:34.636913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c0e46-c595-42c2-a18a-ca4ce3a16d5c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:34.641788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:34.642427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:34.642712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:34.642991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] PATH_INFO: `/v3/users/ed407052035d4863a29bc2254d38195d` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:34.644719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed407052035d4863a29bc2254d38195d)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:34.646631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:34.670191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 937/1884] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:34 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:34.674425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:34.684969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289bba7-dbbe-4237-84ac-116174e7973d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:34.691638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:34.692254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:34.692566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:34.692863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:34.693495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:34.695078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:34.716518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 938/1885] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:34.719405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:34.733329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb6e4e-0c83-4d95-87e6-1b4150635061 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:34.746426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e332dd8-2340-4e78-afd7-6a5e8f665d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:34.747151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e332dd8-2340-4e78-afd7-6a5e8f665d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:34.747433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e332dd8-2340-4e78-afd7-6a5e8f665d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:34.747704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e332dd8-2340-4e78-afd7-6a5e8f665d54 admin admin] PATH_INFO: `/v3/projects/91cd316499264c6e8b000842b55ac6a1/users/d26a9f05c88744af9fd431d600730c4d/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:34.796058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e332dd8-2340-4e78-afd7-6a5e8f665d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cd316499264c6e8b000842b55ac6a1, user_id=d26a9f05c88744af9fd431d600730c4d, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:34.796058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e332dd8-2340-4e78-afd7-6a5e8f665d54 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:35.183936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 939/1886] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:34 2022] PUT /identity/v3/projects/91cd316499264c6e8b000842b55ac6a1/users/d26a9f05c88744af9fd431d600730c4d/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:35.186080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ae4fe-1c58-43ed-b8d3-2e47147e4b8d None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.186381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ae4fe-1c58-43ed-b8d3-2e47147e4b8d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.186846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ae4fe-1c58-43ed-b8d3-2e47147e4b8d None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.188283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 940/1887] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:04:35.194648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ae4fe-1c58-43ed-b8d3-2e47147e4b8d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:35.236591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:35.255862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed407052035d4863a29bc2254d38195d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:35.273435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed407052035d4863a29bc2254d38195d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:35.278811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed407052035d4863a29bc2254d38195d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:35.278811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed407052035d4863a29bc2254d38195d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:35.280148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ae4fe-1c58-43ed-b8d3-2e47147e4b8d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:35.290573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf05f-217c-44db-9243-a5f81db03152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:35.291576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf05f-217c-44db-9243-a5f81db03152 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.291948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf05f-217c-44db-9243-a5f81db03152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.292122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf05f-217c-44db-9243-a5f81db03152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.292818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf05f-217c-44db-9243-a5f81db03152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:35.295047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf05f-217c-44db-9243-a5f81db03152 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:35.312290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 941/1888] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:35.312290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.312290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.312290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.349441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:35.350683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.385549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:35.433312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 942/1889] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:04:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:35.440160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:35.443117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.458274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c25b-46ba-4b5e-8381-c9ab350b1057 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:35.462976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533471b-1be2-48a0-8103-133499e92aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:35.463641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533471b-1be2-48a0-8103-133499e92aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.463985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533471b-1be2-48a0-8103-133499e92aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.464263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533471b-1be2-48a0-8103-133499e92aaa admin admin] PATH_INFO: `/v3/projects/91cd316499264c6e8b000842b55ac6a1/users/d26a9f05c88744af9fd431d600730c4d/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.466260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533471b-1be2-48a0-8103-133499e92aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cd316499264c6e8b000842b55ac6a1, user_id=d26a9f05c88744af9fd431d600730c4d, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:35.470632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533471b-1be2-48a0-8103-133499e92aaa admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:35.480136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 943/1890] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:35 2022] PUT /identity/v3/projects/91cd316499264c6e8b000842b55ac6a1/users/d26a9f05c88744af9fd431d600730c4d/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:35.491574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed407052035d4863a29bc2254d38195d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:35.491735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6609e52-aa4c-45ea-be6a-ed12f760b277 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.491998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6609e52-aa4c-45ea-be6a-ed12f760b277 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.492270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6609e52-aa4c-45ea-be6a-ed12f760b277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.513993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 948/1891] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:34 2022] DELETE /identity/v3/users/ed407052035d4863a29bc2254d38195d => generated 0 bytes in 893 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:35.517364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:35.518811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.533293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b6609e52-aa4c-45ea-be6a-ed12f760b277 None None] MFA Rules not processed for user `d26a9f05c88744af9fd431d600730c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:35.534551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b6609e52-aa4c-45ea-be6a-ed12f760b277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.621418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ebfea-e952-4b90-baea-9515f70bd123 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:35.627795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:35.629189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.629787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.630223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.644424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:35.644424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:35.644766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 944/1892] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:35.651813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.652135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.652450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.658928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 949/1893] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:35.661044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:35.662533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.677059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:35.678301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.724294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761d4-6983-4eed-9afe-2a1de927452d service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:35.728132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:35.728847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.729201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.729484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.739342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:35.740069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 945/1894] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:35.740883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:35.749577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:35.750873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 950/1895] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:35.750997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:35.809186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaa9bb-f764-4dd8-82c7-298caaf6e858 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:35.812874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:35.813481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:35.816150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:35.816150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:35.824558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:35.826084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:35.835638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 946/1896] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:36.004594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:36.013281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1704301-c2e4-4607-9731-0c4c38eb74f4 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:36.016914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:36.017499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:36.017778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:36.018050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:36.020122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:36.078609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:36.079951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:36.089875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 951/1897] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:36.830052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:36.840518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de65f95-2113-4af7-b0d9-c2b5218d06b9 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:36.855943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:36.856625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:36.856913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:36.857180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] PATH_INFO: `/v3/users/623c092ff1314a66b11362582951abd6` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:36.859953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=623c092ff1314a66b11362582951abd6)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:36.862449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:37.120492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:37.123143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:37.139392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d150-7618-4cbe-a87c-f8c106e50796 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:37.144874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:37.146307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:37.146598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:37.146880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] PATH_INFO: `/v3/projects/76d27266cf0d45a9b830b5fcd107ea5f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:37.148068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d27266cf0d45a9b830b5fcd107ea5f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:37.149998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:37.308218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '623c092ff1314a66b11362582951abd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.319587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d27266cf0d45a9b830b5fcd107ea5f', 'request_id': 'req-8cc74090-e4d8-46d2-a04c-07c2d4343b95'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.323383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '623c092ff1314a66b11362582951abd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.464051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d27266cf0d45a9b830b5fcd107ea5f', 'request_id': 'req-8cc74090-e4d8-46d2-a04c-07c2d4343b95'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.469724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d27266cf0d45a9b830b5fcd107ea5f', 'request_id': 'req-8cc74090-e4d8-46d2-a04c-07c2d4343b95'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.481602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] The token cache is being invalidate because project 76d27266cf0d45a9b830b5fcd107ea5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:37.481892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.482541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.484053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 952/1898] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:37 2022] DELETE /identity/v3/projects/76d27266cf0d45a9b830b5fcd107ea5f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:37.488780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:37.490230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:37.566361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74090-e4d8-46d2-a04c-07c2d4343b95 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:37.571910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '623c092ff1314a66b11362582951abd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.572558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '623c092ff1314a66b11362582951abd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.572864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:37.573804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:37.574230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:37.574637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] PATH_INFO: `/v3/users/65b7c5df908545fd8a5d64593b330707` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:37.576502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b7c5df908545fd8a5d64593b330707)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:37.579101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:37.585769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a77bb5fd-06cb-4255-a73a-5bd54abf8cd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '623c092ff1314a66b11362582951abd6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.591787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 947/1899] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:36 2022] DELETE /identity/v3/users/623c092ff1314a66b11362582951abd6 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:37.596132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:37.596515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:37.597701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:37.618884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:37.622979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:37.680560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 948/1900] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:37.690164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:37.691561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:37.755157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af28ce5-3a33-48bb-a516-32a3cc197f4c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:37.759010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:37.759643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:37.759928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:37.760334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:37.770446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:37.772517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:37.786704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 949/1901] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:37.816473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b7c5df908545fd8a5d64593b330707'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.825823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b7c5df908545fd8a5d64593b330707'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.826605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b7c5df908545fd8a5d64593b330707'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.827574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b7c5df908545fd8a5d64593b330707'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:37.890330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:37.892659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:37.934063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825703d6-9acf-4b9b-87fb-6ecda9a8a74b service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:37.938949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:37.939585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:37.939967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:37.940256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] PATH_INFO: `/v3/users/027c865899e74daa90952968bd230f68` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:37.942542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] RBAC: Authorizing `identity:delete_user(user_id=027c865899e74daa90952968bd230f68)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:37.944510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:38.124985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b7c5df908545fd8a5d64593b330707'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.147470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 953/1902] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:37 2022] DELETE /identity/v3/users/65b7c5df908545fd8a5d64593b330707 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:38.246527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027c865899e74daa90952968bd230f68'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.262026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027c865899e74daa90952968bd230f68'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.341797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027c865899e74daa90952968bd230f68'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.346116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027c865899e74daa90952968bd230f68'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.353075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:38.355852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:38.364612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ff8bba1-6e40-484a-acce-a015f4c162de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027c865899e74daa90952968bd230f68'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.376371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 950/1903] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:37 2022] DELETE /identity/v3/users/027c865899e74daa90952968bd230f68 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:38.394794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:38.395696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:38.409633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:38.462149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:38.463361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:38.469869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6a79-cc3f-4e9f-9c3b-f1be116a4cd0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:38.532859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 951/1904] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:38.538315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:38.539032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:38.539387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:38.539736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] PATH_INFO: `/v3/projects/9c733dc7bd18452c9132b562e7481e42` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:38.540974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c733dc7bd18452c9132b562e7481e42)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:38.542546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:38.543017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:38.544035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:38.613073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92de16-d0f6-4723-aea8-3f7fc219bded None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:38.616934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:38.617552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:38.617835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:38.618110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:38.621986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:38.629293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:38.631058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:38.642563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 952/1905] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:38.685038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c733dc7bd18452c9132b562e7481e42', 'request_id': 'req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.690312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:38.787295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c733dc7bd18452c9132b562e7481e42', 'request_id': 'req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.794151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c733dc7bd18452c9132b562e7481e42', 'request_id': 'req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.834150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] The token cache is being invalidate because project 9c733dc7bd18452c9132b562e7481e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:38.834150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.834150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:38.834150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 954/1906] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:38 2022] DELETE /identity/v3/projects/9c733dc7bd18452c9132b562e7481e42 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:38.834653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:38.834653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:38.834653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a221c-9292-4da9-80e3-f4bdecc39a3d service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:38.888206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:38.888886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:38.889165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:38.889439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] PATH_INFO: `/v3/users/d26a9f05c88744af9fd431d600730c4d` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:38.890596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26a9f05c88744af9fd431d600730c4d)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:38.892757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:38.905624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:38.907425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:38.926396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1a999-8ae0-451c-9f6c-93a7b79157b2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:38.931649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:38.932257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:38.933070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:38.933355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] PATH_INFO: `/v3/projects/fff6ccefd04b4955b0b14fc817fde974` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:38.934717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff6ccefd04b4955b0b14fc817fde974)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:38.937428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:39.208671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff6ccefd04b4955b0b14fc817fde974', 'request_id': 'req-f6b49994-f374-47e3-be1b-587e348deb9f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.215756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26a9f05c88744af9fd431d600730c4d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.225485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26a9f05c88744af9fd431d600730c4d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.325184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff6ccefd04b4955b0b14fc817fde974', 'request_id': 'req-f6b49994-f374-47e3-be1b-587e348deb9f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.332535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff6ccefd04b4955b0b14fc817fde974', 'request_id': 'req-f6b49994-f374-47e3-be1b-587e348deb9f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.345746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] The token cache is being invalidate because project fff6ccefd04b4955b0b14fc817fde974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:39.346044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.347264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.349110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 955/1907] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:38 2022] DELETE /identity/v3/projects/fff6ccefd04b4955b0b14fc817fde974 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:39.353574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:39.355175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:39.398785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26a9f05c88744af9fd431d600730c4d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.399434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26a9f05c88744af9fd431d600730c4d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.415830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3aa0a526-2cf5-400d-b5b3-736ee78f6318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26a9f05c88744af9fd431d600730c4d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.423933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 953/1908] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:38 2022] DELETE /identity/v3/users/d26a9f05c88744af9fd431d600730c4d => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:39.426045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc3a5-6efd-4e59-9878-ae327f40923e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:39.426429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc3a5-6efd-4e59-9878-ae327f40923e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:39.426808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efc3a5-6efd-4e59-9878-ae327f40923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:39.447744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b49994-f374-47e3-be1b-587e348deb9f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:39.458447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-46efc3a5-6efd-4e59-9878-ae327f40923e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:39.458447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-46efc3a5-6efd-4e59-9878-ae327f40923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:39.569768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 954/1909] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:39.571454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:39.571822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:39.572173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:39.597277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:39.597991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:39.598873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:39.599027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:39.599221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:39.599505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] PATH_INFO: `/v3/projects/1dac52ef2df542a18fee84878e22ffc6` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:39.600863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dac52ef2df542a18fee84878e22ffc6)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:39.603191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:39.614245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 955/1910] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:39.615341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:39.627983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33735437-f2a1-4ab8-8943-756af13e9fd6 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:39.634534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:39.635442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:39.636013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:39.636426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:39.637202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:39.637607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:39.640152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:39.662673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 956/1911] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:39.667607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:39.689598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2333ff-0736-43d6-a0c7-8c20d72d0366 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:39.697559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:39.698888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:39.699414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:39.700042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:39.701335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:39.705175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:39.777988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dac52ef2df542a18fee84878e22ffc6', 'request_id': 'req-19e241ae-2070-4b9e-a8c1-60ac69e6695c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.840369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 957/1912] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:39 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:39.844231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:39.906017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bf352-3cfc-4114-80f8-a58393389398 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:39.913278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:39.914063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:39.914412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:39.914755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:39.915683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:39.917854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:39.941692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dac52ef2df542a18fee84878e22ffc6', 'request_id': 'req-19e241ae-2070-4b9e-a8c1-60ac69e6695c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.951840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dac52ef2df542a18fee84878e22ffc6', 'request_id': 'req-19e241ae-2070-4b9e-a8c1-60ac69e6695c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.964518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] The token cache is being invalidate because project 1dac52ef2df542a18fee84878e22ffc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:39.965022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.965787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:39.968757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 956/1913] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:39 2022] DELETE /identity/v3/projects/1dac52ef2df542a18fee84878e22ffc6 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:39.972984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:39.977333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.043427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 958/1914] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:39 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:40.047903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:40.049426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.053389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e241ae-2070-4b9e-a8c1-60ac69e6695c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.058507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.059284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.059629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.059974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] PATH_INFO: `/v3/users/fedbdf68b9e741968d88f9c531c7a0d0` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.065878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a01fb-aa01-4c1d-a2be-7207b2f11567 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.077306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.080215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.080215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.080215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.080643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fedbdf68b9e741968d88f9c531c7a0d0)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:40.080831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:40.083736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:40.084591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:40.117450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 959/1915] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:40.118564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:40.131414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbfc5a-cbd6-40a4-98e6-b272f8c45853 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.136591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.137559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.138042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.138559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.139422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:40.141319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:40.156296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 960/1916] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:40.157631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:40.168965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fa671-501b-4fed-8611-b0bf0608fba3 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.174028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.174768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.175057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.175386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] PATH_INFO: `/v3/projects/68628da9dfef45c8964e0ea546d37270/users/f1724a8c8b764ab2b828cf32b9299c1c/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.190240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] RBAC: Authorizing `identity:create_grant(project_id=68628da9dfef45c8964e0ea546d37270, user_id=f1724a8c8b764ab2b828cf32b9299c1c, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:40.193839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:40.309284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 961/1917] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:40 2022] PUT /identity/v3/projects/68628da9dfef45c8964e0ea546d37270/users/f1724a8c8b764ab2b828cf32b9299c1c/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:40.319694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:40.321469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.400415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81c45a-d652-499c-b14c-88a237e3991e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.405236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.405946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.406223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.406797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] PATH_INFO: `/v3/projects/91cd316499264c6e8b000842b55ac6a1` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.407930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cd316499264c6e8b000842b55ac6a1)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:40.410077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:40.482330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fedbdf68b9e741968d88f9c531c7a0d0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.490590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fedbdf68b9e741968d88f9c531c7a0d0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.491588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fedbdf68b9e741968d88f9c531c7a0d0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.492221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fedbdf68b9e741968d88f9c531c7a0d0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.579471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cd316499264c6e8b000842b55ac6a1', 'request_id': 'req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.598352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-53ccf5a7-f92b-41ce-9e60-d90774d6f844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fedbdf68b9e741968d88f9c531c7a0d0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.617319 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 957/1918] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:39 2022] DELETE /identity/v3/users/fedbdf68b9e741968d88f9c531c7a0d0 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:40.618734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.619021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.619294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.649131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] MFA Rules not processed for user `f1724a8c8b764ab2b828cf32b9299c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:40.650350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.684517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cd316499264c6e8b000842b55ac6a1', 'request_id': 'req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.702928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cd316499264c6e8b000842b55ac6a1', 'request_id': 'req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.728388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 958/1919] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:40.735097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] The token cache is being invalidate because project 91cd316499264c6e8b000842b55ac6a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:40.735397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.740789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-83515c77-e1cc-4aca-bf46-26a5b3c4cd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:40.749255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:40.749255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.749539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 962/1920] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:40 2022] DELETE /identity/v3/projects/91cd316499264c6e8b000842b55ac6a1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:40.752768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d275590-7994-4490-83a4-3c5abf7f3a7a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.753057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d275590-7994-4490-83a4-3c5abf7f3a7a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.753394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d275590-7994-4490-83a4-3c5abf7f3a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.789464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6d275590-7994-4490-83a4-3c5abf7f3a7a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:40.790635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6d275590-7994-4490-83a4-3c5abf7f3a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.865563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85b6af-2d24-4a63-b18f-21240abe1ef4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.873512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.874930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.875373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.876013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.895164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.897620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 963/1921] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:40.905848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:40.906272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:40.907075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:40.918980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:40.920849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:40.933435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:40.935456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:40.939748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 959/1922] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:40.950909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 964/1923] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:40.954159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:40.988494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416d123-94f7-4796-9272-79a4f78b2332 service glance] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:40.994080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:40.995693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.002760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.002760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.002760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:41.002760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.002760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.018386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 960/1924] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:41.023282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.035808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ed58c-7fa4-44b7-ad1e-8974c90a46f7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.051817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.052485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.052768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.053039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.053681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.060294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.162925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:41.202697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 965/1925] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:41 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:41.206831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.219667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1dbdd-74fe-484d-add0-3863add24b96 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.325911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.327336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.327336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.327336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.328527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.330520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.401836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.403546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:41.419103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7778-0736-436f-8fa1-72ff839806a7 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.429589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 961/1926] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:41 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:41.432927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.433562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.434362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.438893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.439321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] PATH_INFO: `/v3/projects/305cb5c10f694c5683c8d79bbaf9c038` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.445044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b27b5-1696-4af7-ab01-69d332a840e2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.445848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=305cb5c10f694c5683c8d79bbaf9c038)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.448788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.450806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.451223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.451517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.451792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.465906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.465906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.479759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 962/1927] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:41.483760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.518718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f7c1-1b7c-49bb-a843-c6c7abc499aa admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.525834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.526746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.527246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.527697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.529286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.531082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.548802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 963/1928] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:41.553292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.565112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360047a-0fc6-42c9-89ca-f7ab6e39c3bc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.570936 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9185910-e84f-4dfb-bc3f-ea3acb73e5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.571982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9185910-e84f-4dfb-bc3f-ea3acb73e5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.572415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9185910-e84f-4dfb-bc3f-ea3acb73e5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.572718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9185910-e84f-4dfb-bc3f-ea3acb73e5fd admin admin] PATH_INFO: `/v3/projects/99fad5e0b9aa4e3a8a69bcc270caf99b/users/d87d9ed756014d73a8bc326221e97e94/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.604171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9185910-e84f-4dfb-bc3f-ea3acb73e5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fad5e0b9aa4e3a8a69bcc270caf99b, user_id=d87d9ed756014d73a8bc326221e97e94, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.604867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9185910-e84f-4dfb-bc3f-ea3acb73e5fd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.617716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305cb5c10f694c5683c8d79bbaf9c038', 'request_id': 'req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:41.692737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 964/1929] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:41 2022] PUT /identity/v3/projects/99fad5e0b9aa4e3a8a69bcc270caf99b/users/d87d9ed756014d73a8bc326221e97e94/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:41.697555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.697843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.698120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.716573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:41.717766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:41.737777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305cb5c10f694c5683c8d79bbaf9c038', 'request_id': 'req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:41.747076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305cb5c10f694c5683c8d79bbaf9c038', 'request_id': 'req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:41.767214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] The token cache is being invalidate because project 305cb5c10f694c5683c8d79bbaf9c038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:41.767803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:41.769985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d7fefcb-ba61-429d-8bc0-b00cc4e952cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:41.773657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 966/1930] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:41 2022] DELETE /identity/v3/projects/305cb5c10f694c5683c8d79bbaf9c038 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:41.788823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9717bef-c1ed-4048-b254-0a016258e41e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.789252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9717bef-c1ed-4048-b254-0a016258e41e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.789425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9717bef-c1ed-4048-b254-0a016258e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.803740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 965/1931] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:41.810873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c9717bef-c1ed-4048-b254-0a016258e41e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:41.812115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c9717bef-c1ed-4048-b254-0a016258e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:41.815492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.817991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:41.827165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 967/1932] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:41.834234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.834575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.834885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.857049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:41.858075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:41.870210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 968/1933] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:41.874310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.887998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad1667-7b56-43cd-982c-d30de5eb0e45 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.888747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad34cb-f2ff-4660-88ae-9f30f95f5961 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.897052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.897376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.898554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.898848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.899526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:41.899831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.901653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.901816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.902305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.902642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.902929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.913818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.915266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:41.915507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 969/1934] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:04:41.920167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:41.931611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb2cb3-c287-41e4-a977-1f31d6754aa6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:41.936679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 966/1935] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:41.937377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:41.938209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:41.938566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:41.938910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:41.939634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:41.942778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:42.005267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:42.078402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 970/1936] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:41 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:42.083463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:42.098921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6824f3-feb3-4bc0-8203-1b9bd55ea940 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:42.186308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:42.189073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:42.189073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:42.189073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:42.189073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:42.193830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:42.337777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 967/1937] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:42 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:42.350449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:42.378108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d868e7f-ae32-430f-ab8f-9f985f0b64e0 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:42.386881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:42.388074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:42.388231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:42.388981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:42.389938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:42.393141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:42.416947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 971/1938] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:42.418792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:42.432839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38c9d8-cf15-4819-a6a1-246667cd7e0e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:42.437897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046db05-8ff4-490c-9c49-5981f2808054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:42.438627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046db05-8ff4-490c-9c49-5981f2808054 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:42.438916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046db05-8ff4-490c-9c49-5981f2808054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:42.439196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046db05-8ff4-490c-9c49-5981f2808054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:42.439949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046db05-8ff4-490c-9c49-5981f2808054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:42.442188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046db05-8ff4-490c-9c49-5981f2808054 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:42.456230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 968/1939] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:04:42.458133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:42.469057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2bef8-bd64-4ca3-873f-b01aa9c26d8c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:42.483780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:42.484722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:42.486120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:42.486549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] PATH_INFO: `/v3/projects/d878eb99274c4d1da42c1f880edb77ad/users/bdbf64c172984ace8ec68987f56837cb/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:42.506624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d878eb99274c4d1da42c1f880edb77ad, user_id=bdbf64c172984ace8ec68987f56837cb, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:42.509460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:42.595550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 972/1940] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:42 2022] PUT /identity/v3/projects/d878eb99274c4d1da42c1f880edb77ad/users/bdbf64c172984ace8ec68987f56837cb/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:42.608296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:42.608778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:42.608876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:42.646337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] MFA Rules not processed for user `bdbf64c172984ace8ec68987f56837cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:42.648713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:42.747891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:42.748075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 969/1941] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:42.904396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfcae4-dca5-4624-a19c-8311eab6ab3d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:42.912096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:42.913084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:42.913632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:42.914623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:42.915450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:42.918077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:43.096372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 973/1942] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:42 2022] POST /identity/v3/projects => generated 379 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:43.097565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:43.108268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86ea6-1827-466c-b341-7c4dc2c20422 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:43.174100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:43.174768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.175075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.175903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.176688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:43.178891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:43.411615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 970/1943] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:43 2022] POST /identity/v3/users => generated 296 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:43.414146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:43.431194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216de930-7a27-4ab5-b6f9-08bdaa2f9099 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:43.436627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:43.437454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.437796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.438074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.438739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:43.440475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:43.457289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 974/1944] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:43.457995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:43.471336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358639d-1a40-463c-9f17-081da6d749d5 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:43.484286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:43.484286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.484286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.486594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] PATH_INFO: `/v3/projects/e11009cedc3343e69e08f276c6b4859e/users/7b8caff028f44854a9ab68cd820abe2f/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.507440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11009cedc3343e69e08f276c6b4859e, user_id=7b8caff028f44854a9ab68cd820abe2f, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:43.510414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:43.693287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:43.693654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 971/1945] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:43 2022] PUT /identity/v3/projects/e11009cedc3343e69e08f276c6b4859e/users/7b8caff028f44854a9ab68cd820abe2f/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:43.707736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1be38c-0efe-4436-811e-0383851f0f5c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:43.773351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704833a-769a-48a9-be65-96576c6c01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:43.774174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704833a-769a-48a9-be65-96576c6c01d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.774344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704833a-769a-48a9-be65-96576c6c01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.774675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704833a-769a-48a9-be65-96576c6c01d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.775141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704833a-769a-48a9-be65-96576c6c01d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:43.777883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704833a-769a-48a9-be65-96576c6c01d6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:43.791854 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 975/1946] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:43.793446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:43.804944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fba596-dfbc-44b7-9ae2-5d9cfb29ecda admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:43.823723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60f72-ed27-4633-adce-92672f58f0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:43.823723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60f72-ed27-4633-adce-92672f58f0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.823723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60f72-ed27-4633-adce-92672f58f0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.823723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60f72-ed27-4633-adce-92672f58f0b5 admin admin] PATH_INFO: `/v3/projects/e11009cedc3343e69e08f276c6b4859e/users/7b8caff028f44854a9ab68cd820abe2f/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.824303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60f72-ed27-4633-adce-92672f58f0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11009cedc3343e69e08f276c6b4859e, user_id=7b8caff028f44854a9ab68cd820abe2f, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:43.824303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60f72-ed27-4633-adce-92672f58f0b5 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:43.842359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 972/1947] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:43 2022] PUT /identity/v3/projects/e11009cedc3343e69e08f276c6b4859e/users/7b8caff028f44854a9ab68cd820abe2f/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:43.856819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.857105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.857902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.895894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] MFA Rules not processed for user `7b8caff028f44854a9ab68cd820abe2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:43.898178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:43.926137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:43.926571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:43.926734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:43.978966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] MFA Rules not processed for user `d87d9ed756014d73a8bc326221e97e94`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:43.980776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:44.014457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 976/1948] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:44.027989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.029550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:44.071802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 973/1949] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:44.075873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.077248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:44.147051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa8f7-29b2-41fe-a3ca-7f0c691f62a8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.150845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.152167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.152766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.152766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.166882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.168554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.181435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adbfda-e6c5-4e60-89a5-39a58052755c None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.182912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 977/1950] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:04:44.191255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.191951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.192237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.192572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.193233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.195118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.388960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 974/1951] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:44 2022] POST /identity/v3/projects => generated 371 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:04:44.393017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.405130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2a257-f7f9-406e-a186-544ffc4e09e7 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.466111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.466944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.467383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.467861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.468820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.471158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.524579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:44.564807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 978/1952] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:44 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:44.574918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.611799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16421e4-a572-4ca7-bb92-f72af6266289 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.616784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.617619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.617996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.618366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.619189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.620985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.645166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 975/1953] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:44.647767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.658281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95722-200a-45d4-a3b1-17b7398e517d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.668265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.668846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.669137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.669410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] PATH_INFO: `/v3/projects/7fea72bddc6044599a875528ea4eaec7/users/2708c116cb1f4631bf0722e1145ce23e/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.686358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fea72bddc6044599a875528ea4eaec7, user_id=2708c116cb1f4631bf0722e1145ce23e, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.694905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.790308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 979/1954] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:44 2022] PUT /identity/v3/projects/7fea72bddc6044599a875528ea4eaec7/users/2708c116cb1f4631bf0722e1145ce23e/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:44.794580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.805689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799654-1b39-4e61-acb7-8d99592c6c12 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.866621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.867234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.867515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.868121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.868866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.870443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.884443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 976/1955] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:44.885545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.892016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:44.899368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f80d1-913b-488a-9c27-74a194d2fe9d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.901172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d234900-5d43-4961-8db1-4695b8718c9a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:44.904741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90aba3-0500-4f5f-a88e-c0b765ea7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.905507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90aba3-0500-4f5f-a88e-c0b765ea7ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.905787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90aba3-0500-4f5f-a88e-c0b765ea7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.906067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90aba3-0500-4f5f-a88e-c0b765ea7ee2 admin admin] PATH_INFO: `/v3/projects/7fea72bddc6044599a875528ea4eaec7/users/2708c116cb1f4631bf0722e1145ce23e/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.906280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:44.907170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:44.907448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:44.907756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] PATH_INFO: `/v3/users/bdbf64c172984ace8ec68987f56837cb` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:44.908648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90aba3-0500-4f5f-a88e-c0b765ea7ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fea72bddc6044599a875528ea4eaec7, user_id=2708c116cb1f4631bf0722e1145ce23e, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.908917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbf64c172984ace8ec68987f56837cb)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:44.911562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.916704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90aba3-0500-4f5f-a88e-c0b765ea7ee2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:44.934213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 980/1956] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:44 2022] PUT /identity/v3/projects/7fea72bddc6044599a875528ea4eaec7/users/2708c116cb1f4631bf0722e1145ce23e/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:45.139252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbf64c172984ace8ec68987f56837cb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:45.146292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbf64c172984ace8ec68987f56837cb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:45.147075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbf64c172984ace8ec68987f56837cb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:45.147681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbf64c172984ace8ec68987f56837cb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:45.244779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbf64c172984ace8ec68987f56837cb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:45.285348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:45.293580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 977/1957] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:44 2022] DELETE /identity/v3/users/bdbf64c172984ace8ec68987f56837cb => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:04:45.298247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:45.298807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:45.299851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:45.323639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:45.324996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:45.424014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 981/1958] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:45.446646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:45.447583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:45.513807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee720bc-75e0-49dd-9b6e-8cdc902cfc38 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:45.517726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684080-2194-475c-b661-775d583c5824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:45.518443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684080-2194-475c-b661-775d583c5824 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:45.519525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684080-2194-475c-b661-775d583c5824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:45.519799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684080-2194-475c-b661-775d583c5824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:45.529631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41684080-2194-475c-b661-775d583c5824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:45.531044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41684080-2194-475c-b661-775d583c5824 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:45.541407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 978/1959] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:46.139277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:46.144379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:46.154893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47832ec4-245c-46ab-be00-ea7b49943bd1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:46.160976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:46.161786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:46.162165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:46.162932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] PATH_INFO: `/v3/projects/d878eb99274c4d1da42c1f880edb77ad` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:46.164558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d878eb99274c4d1da42c1f880edb77ad)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:46.166941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:46.385995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd878eb99274c4d1da42c1f880edb77ad', 'request_id': 'req-f49748f0-5bda-43d0-b539-0a4bad93daa7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.463785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd878eb99274c4d1da42c1f880edb77ad', 'request_id': 'req-f49748f0-5bda-43d0-b539-0a4bad93daa7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.469879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd878eb99274c4d1da42c1f880edb77ad', 'request_id': 'req-f49748f0-5bda-43d0-b539-0a4bad93daa7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.481785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] The token cache is being invalidate because project d878eb99274c4d1da42c1f880edb77ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:46.482076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.482702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f49748f0-5bda-43d0-b539-0a4bad93daa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.484203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 982/1960] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:46 2022] DELETE /identity/v3/projects/d878eb99274c4d1da42c1f880edb77ad => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:46.488575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684080-2194-475c-b661-775d583c5824 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:46.498160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684080-2194-475c-b661-775d583c5824 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:46.560975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:46.565063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:46.565063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:46.565063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] PATH_INFO: `/v3/users/7b8caff028f44854a9ab68cd820abe2f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:46.565063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8caff028f44854a9ab68cd820abe2f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:46.565506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:46.814141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8caff028f44854a9ab68cd820abe2f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.820903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8caff028f44854a9ab68cd820abe2f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.821583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8caff028f44854a9ab68cd820abe2f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.822191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8caff028f44854a9ab68cd820abe2f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.847979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:46.848264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:46.848640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:46.875982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] MFA Rules not processed for user `2708c116cb1f4631bf0722e1145ce23e`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:46.877228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:46.929841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8caff028f44854a9ab68cd820abe2f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:46.930172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:46.952348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 979/1961] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:04:46 2022] DELETE /identity/v3/users/7b8caff028f44854a9ab68cd820abe2f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:46.961762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 983/1962] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:46.992695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:46.992695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:47.051967 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902b629-c20b-48d0-bea7-ad9c69813cc8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:47.055786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:47.056517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:47.056801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:47.057131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:47.058904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:47.116811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:47.118230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f4c05-756c-4534-b716-a7f3600b7aa9 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:47.126966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 980/1963] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:47.437295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:47.439688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:47.454070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff249d47-c419-4ee5-ae6b-62a5420cc871 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:47.470040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:47.510903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:47.511559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:47.511835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:47.512101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] PATH_INFO: `/v3/projects/e11009cedc3343e69e08f276c6b4859e` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:47.513246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11009cedc3343e69e08f276c6b4859e)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:47.515215 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:47.649429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11009cedc3343e69e08f276c6b4859e', 'request_id': 'req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:47.770501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11009cedc3343e69e08f276c6b4859e', 'request_id': 'req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:47.779349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11009cedc3343e69e08f276c6b4859e', 'request_id': 'req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:47.793604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] The token cache is being invalidate because project e11009cedc3343e69e08f276c6b4859e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:04:47.794162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:47.795144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-ead5ac7a-3aa2-4085-be1f-47b84b7e1f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:04:47.797833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 984/1964] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:04:47 2022] DELETE /identity/v3/projects/e11009cedc3343e69e08f276c6b4859e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:47.802917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:47.803475 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:47.803653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:47.830303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:47.832539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:47.929356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 981/1965] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:04:47.938044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:47.938333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:47.938608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:47.961818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:47.963147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:47.974183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 985/1966] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:47.979655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:47.989694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c60396-ff21-49c5-9e78-7a3c3990108a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:47.996022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:47.996022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:47.996022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:47.996022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:47.998704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:04:47.999025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.000656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.019949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 982/1967] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:04:48.029391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:48.039930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867e9eb-c309-45d1-a343-4f25dbd4e721 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:48.045178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:48.045802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.046082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.046357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.047194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.049095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.157079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 986/1968] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:04:48 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:48.161449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:48.171574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161163b-2e00-4c7c-9ead-84306ff16336 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:48.227519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:48.228070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.228377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.228659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.229287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.231223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.343814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 983/1969] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:04:48 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:04:48.347978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:48.358341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47726f8a-bd0b-4a95-9e1d-e24eb934a728 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:48.363583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:48.364194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.364497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.364822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.365476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.367136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.377080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 987/1970] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:48.381814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:48.391973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fad00-a9f3-4cbd-b30c-94767542620e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:48.396747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740eff90-e9b8-4275-9db0-9e8f6e0ece3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:48.397350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eff90-e9b8-4275-9db0-9e8f6e0ece3a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.397773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eff90-e9b8-4275-9db0-9e8f6e0ece3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.398053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eff90-e9b8-4275-9db0-9e8f6e0ece3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.398704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740eff90-e9b8-4275-9db0-9e8f6e0ece3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.400284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740eff90-e9b8-4275-9db0-9e8f6e0ece3a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.412728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 984/1971] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:04:48.417508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:48.428159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab5e3-7d2e-42a5-9691-ee359c527148 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:48.432698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:48.433339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.433617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.433888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] PATH_INFO: `/v3/projects/5e6cfc1041dd490fb95c476bd5fe0b44/users/ad6e2388ca3d404291b5f8c1ac5594b8/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.455209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6cfc1041dd490fb95c476bd5fe0b44, user_id=ad6e2388ca3d404291b5f8c1ac5594b8, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.455209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.549152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 988/1972] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:04:48 2022] PUT /identity/v3/projects/5e6cfc1041dd490fb95c476bd5fe0b44/users/ad6e2388ca3d404291b5f8c1ac5594b8/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:04:48.563356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.563356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.563356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.587992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] MFA Rules not processed for user `ad6e2388ca3d404291b5f8c1ac5594b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:04:48.589229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:48.652617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 985/1973] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:04:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:04:48.664565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:48.666879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:48.726091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bcb1f-496f-45be-b9d0-bf41006209af admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:48.729871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:48.730478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:48.730760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:48.731035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:48.739298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:48.741457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:48.742900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:48.752823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 989/1974] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:49.398872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:49.587200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:50.647011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:50.698096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:52.215470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:04:52.217082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:52.229884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ee296-9de6-4cfc-98b1-4a38a8103553 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:04:52.233504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:04:52.234109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:04:52.237431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:04:52.237431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:04:52.237431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:04:52.259839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:52.310920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:04:52.312373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b7973-f457-44b2-91ee-58244f3de17a service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:04:52.321856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 986/1975] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:04:52.358336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:53.478953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:53.530876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:54.727236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:55.227005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:55.530412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:55.730169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:55.992772 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:56.258039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:56.304196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:56.547321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:56.672744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:56.989586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.061162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.228789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.252612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.505129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.532493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.669233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.692567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.843868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:57.930197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.052485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.098462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.233309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.387418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.695571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.875198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:58.987065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:04:59.163901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:07.427931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:07.429385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:07.497591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325b594-cbda-4a81-8469-1c2a3ad6ea96 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:07.502280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:07.503159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:07.503697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:07.503923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] PATH_INFO: `/v3/users/4449dcfa2ed244eb831cc2fbf7d7ef4f` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:07.505103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4449dcfa2ed244eb831cc2fbf7d7ef4f)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:07.507057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:07.528558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:INSERT=3,keystone:DELETE=5,keystone:SELECT=92 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:07.754275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4449dcfa2ed244eb831cc2fbf7d7ef4f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:07.760984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4449dcfa2ed244eb831cc2fbf7d7ef4f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:07.821111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4449dcfa2ed244eb831cc2fbf7d7ef4f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:07.821738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4449dcfa2ed244eb831cc2fbf7d7ef4f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:07.837476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4449dcfa2ed244eb831cc2fbf7d7ef4f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:07.843756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 990/1976] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:07 2022] DELETE /identity/v3/users/4449dcfa2ed244eb831cc2fbf7d7ef4f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:07.848991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:07.849485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:07.849763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:07.869449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:07.872464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:07.890294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:07.930047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 987/1977] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:07.940548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:07.942552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:08.013391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e264a-e06b-484c-9100-97a435a529d6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:08.017012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:08.017475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:08.017751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:08.018022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:08.027728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:08.029179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:08.038956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 991/1978] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:08.659769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:08.661504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:08.673972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d14df5-624e-4122-89da-947f13e804c2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:08.679722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:08.680488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:08.680774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:08.681048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] PATH_INFO: `/v3/projects/8901804e235442578d0e2120fbb2f7ac` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:08.682195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8901804e235442578d0e2120fbb2f7ac)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:08.686627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:08.848891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8901804e235442578d0e2120fbb2f7ac', 'request_id': 'req-977455f3-e1c5-4a28-9b88-2ffa9c568af6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:08.972222 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8901804e235442578d0e2120fbb2f7ac', 'request_id': 'req-977455f3-e1c5-4a28-9b88-2ffa9c568af6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:08.980514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8901804e235442578d0e2120fbb2f7ac', 'request_id': 'req-977455f3-e1c5-4a28-9b88-2ffa9c568af6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.010350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] The token cache is being invalidate because project 8901804e235442578d0e2120fbb2f7ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:09.015121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.016070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.018035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 988/1979] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:08 2022] DELETE /identity/v3/projects/8901804e235442578d0e2120fbb2f7ac => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:09.025446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:09.038144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeb99d-1ddc-4fcd-a557-c8f742cfc0c6 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:09.109668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:09.110329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:09.110696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:09.110918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] PATH_INFO: `/v3/users/be4aed92790e4f638f08aea9c1520b24` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:09.112736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4aed92790e4f638f08aea9c1520b24)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:09.114673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:09.333262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:09.349351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4aed92790e4f638f08aea9c1520b24'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.357010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4aed92790e4f638f08aea9c1520b24'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.461668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4aed92790e4f638f08aea9c1520b24'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.462330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4aed92790e4f638f08aea9c1520b24'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.476909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-962678b1-f216-449e-9010-4e6975027ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4aed92790e4f638f08aea9c1520b24'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:09.483297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 992/1980] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:09 2022] DELETE /identity/v3/users/be4aed92790e4f638f08aea9c1520b24 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:10.330224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:10.331809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:10.372727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977455f3-e1c5-4a28-9b88-2ffa9c568af6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:10.461012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:10.461841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:10.462190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:10.462535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] PATH_INFO: `/v3/projects/c9630a84df434d3aaa5b203e88d69eb5` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:10.463845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9630a84df434d3aaa5b203e88d69eb5)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:10.465926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:10.641258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9630a84df434d3aaa5b203e88d69eb5', 'request_id': 'req-93493f05-2c69-4dc3-9b52-689e71623a3d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:10.644369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:10.741942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9630a84df434d3aaa5b203e88d69eb5', 'request_id': 'req-93493f05-2c69-4dc3-9b52-689e71623a3d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:10.751429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9630a84df434d3aaa5b203e88d69eb5', 'request_id': 'req-93493f05-2c69-4dc3-9b52-689e71623a3d'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:10.761613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] The token cache is being invalidate because project c9630a84df434d3aaa5b203e88d69eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:10.761901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:10.762560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-93493f05-2c69-4dc3-9b52-689e71623a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:10.764076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 989/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:10 2022] DELETE /identity/v3/projects/c9630a84df434d3aaa5b203e88d69eb5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:10.769425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:10.769907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:10.770006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:10.794870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:10.796106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:10.856948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 993/1982] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:10.866318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:10.866736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:10.867097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:10.887840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:10.889140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:10.901340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 990/1983] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:10.903728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:10.914518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bf8ed-f799-44bf-a848-ec98e02c9634 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:10.919132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:10.919678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:10.919955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:10.920230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:10.920923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:10.921224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:10.922910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:10.936249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 994/1984] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:05:10.941593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:10.951070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e007d-209c-4eec-82ae-862c7c5d5359 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:10.956386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:10.956993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:10.957273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:10.957542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:10.958190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:10.960130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:11.134236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 991/1985] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:10 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:05:11.135429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:11.148676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f265d-5898-451d-bd00-11e6a8838fcc admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:11.220339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:11.221019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:11.221357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:11.221632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:11.222271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:11.224221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:11.341919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 995/1986] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:11 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:11.354295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:11.382067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffc7e2-9cee-495e-9bf1-5f13fed35448 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:11.387282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:11.387899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:11.388184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:11.388490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:11.389457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:11.391175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:11.415388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 992/1987] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:11.428591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:11.443350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c80f2f-0955-4cf4-af25-4b5a1d69bc6e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:11.449658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395019e-3993-43e7-a550-8b457cb0074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:11.450437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395019e-3993-43e7-a550-8b457cb0074a admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:11.451577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395019e-3993-43e7-a550-8b457cb0074a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:11.451918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395019e-3993-43e7-a550-8b457cb0074a admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:11.453157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395019e-3993-43e7-a550-8b457cb0074a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:11.454559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395019e-3993-43e7-a550-8b457cb0074a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:11.467109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 996/1988] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:11.471271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:11.482046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e982ab-09ca-45a2-80c2-fe5fa1cc2bb0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:11.486509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:11.487907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:11.488183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:11.488497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] PATH_INFO: `/v3/projects/b5b31cc9493d4bf69d403348cdb319ac/users/6e0708974466456696daf3e5cef3498e/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:11.504021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b31cc9493d4bf69d403348cdb319ac, user_id=6e0708974466456696daf3e5cef3498e, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:11.506730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:11.628158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 993/1989] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:11 2022] PUT /identity/v3/projects/b5b31cc9493d4bf69d403348cdb319ac/users/6e0708974466456696daf3e5cef3498e/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:11.638967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:11.639256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:11.639528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:11.672720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] MFA Rules not processed for user `6e0708974466456696daf3e5cef3498e`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:11.674040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:11.689778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:11.764510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 997/1990] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:11.772682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:11.836983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b24ae-111c-4dd2-af6d-a0889107229e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:11.842904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:11.843526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:11.843813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:11.844086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:11.844766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:11.854848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:11.979977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 994/1991] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:11 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:11.983984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:11.997241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a942-62d3-4dab-843b-3f217923e1eb None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.056658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.057274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.057556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.057838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.058515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.060555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.170955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 998/1992] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:11 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:12.173923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:12.184345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0df5b-0193-4e7e-8112-5afd5f1206ca admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.188948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.189548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.189832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.190138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.190771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.192411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.203108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 995/1993] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:12.207682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:12.217748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d319c8-8de2-41c2-a696-7e579f91ea5c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.223113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.223762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.224043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.224329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] PATH_INFO: `/v3/projects/92f6d860a0904e948df159faf010de58/users/599056493c034b6a839b0620a60042f6/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.239946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f6d860a0904e948df159faf010de58, user_id=599056493c034b6a839b0620a60042f6, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.242561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.327676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 999/1994] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:12 2022] PUT /identity/v3/projects/92f6d860a0904e948df159faf010de58/users/599056493c034b6a839b0620a60042f6/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:12.332188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:12.341987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360caa2f-c276-4fd6-b0ad-ba2f4c10d5a4 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.389712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:12.400451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.401087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.401353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.401630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.402277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.403859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.414803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 996/1995] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:12.418876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:12.429484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343d41c-8904-4239-a840-262a4a6f8b61 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.434308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd36e75-466f-4160-9866-45940778c9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.434968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd36e75-466f-4160-9866-45940778c9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.435328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd36e75-466f-4160-9866-45940778c9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.435600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd36e75-466f-4160-9866-45940778c9ca admin admin] PATH_INFO: `/v3/projects/92f6d860a0904e948df159faf010de58/users/599056493c034b6a839b0620a60042f6/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.437050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:12.443775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd36e75-466f-4160-9866-45940778c9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f6d860a0904e948df159faf010de58, user_id=599056493c034b6a839b0620a60042f6, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.444009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:12.444126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd36e75-466f-4160-9866-45940778c9ca admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.452724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1000/1996] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:12 2022] PUT /identity/v3/projects/92f6d860a0904e948df159faf010de58/users/599056493c034b6a839b0620a60042f6/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:12.454898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362d81b-713e-44a4-a88d-549007d886d7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.465097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.465382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.465656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.501295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] MFA Rules not processed for user `599056493c034b6a839b0620a60042f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:12.503805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:12.527571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.528227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.529327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.529654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] PATH_INFO: `/v3/users/ad6e2388ca3d404291b5f8c1ac5594b8` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.531592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6e2388ca3d404291b5f8c1ac5594b8)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.534607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.574242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:12.592760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1001/1997] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:12.604048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:12.605428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:12.668172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c0539-1f9d-4aac-9af5-2f9b15d73d9d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:12.671812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e3ee0-0f54-439d-976a-617753a0df0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:12.672455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e3ee0-0f54-439d-976a-617753a0df0f service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.672734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e3ee0-0f54-439d-976a-617753a0df0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.673007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e3ee0-0f54-439d-976a-617753a0df0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.682132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e3ee0-0f54-439d-976a-617753a0df0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:12.683574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e3ee0-0f54-439d-976a-617753a0df0f service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:12.694059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1002/1998] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:12.771189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6e2388ca3d404291b5f8c1ac5594b8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:12.777960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6e2388ca3d404291b5f8c1ac5594b8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:12.778637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6e2388ca3d404291b5f8c1ac5594b8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:12.779233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6e2388ca3d404291b5f8c1ac5594b8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:12.866718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6e2388ca3d404291b5f8c1ac5594b8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:12.896504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 997/1999] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:12 2022] DELETE /identity/v3/users/ad6e2388ca3d404291b5f8c1ac5594b8 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:12.901164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:12.901856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:12.902571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:12.937078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:12.937446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:13.005418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:13.006832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:13.012386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1003/2000] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:13.021500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:13.023577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:13.072553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90254b04-cae0-4815-aabc-abe39443e3c2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:13.076336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:13.077118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:13.077464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:13.077790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:13.079780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:13.086639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3561b0-5379-4dcb-8816-6aa59eb4d07f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:13.090940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:13.091567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:13.091850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:13.092126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:13.101856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:13.102417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:13.103841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:13.114031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1004/2001] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:13.143585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:13.145269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de338a45-4d33-472f-95d9-84fb3f0bb4b3 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:13.175231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 998/2002] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:13.597483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:13.598964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:13.618388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364230e1-a02d-4356-982a-fb807ae7f52d service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:13.625169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:13.625717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:13.625915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:13.626194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] PATH_INFO: `/v3/projects/5e6cfc1041dd490fb95c476bd5fe0b44` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:13.631194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6cfc1041dd490fb95c476bd5fe0b44)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:13.633458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:13.773199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:13.785493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6cfc1041dd490fb95c476bd5fe0b44', 'request_id': 'req-f3040631-1d4b-45d7-af9a-32cfe84507a1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:13.879568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6cfc1041dd490fb95c476bd5fe0b44', 'request_id': 'req-f3040631-1d4b-45d7-af9a-32cfe84507a1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:13.885812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6cfc1041dd490fb95c476bd5fe0b44', 'request_id': 'req-f3040631-1d4b-45d7-af9a-32cfe84507a1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:13.900659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] The token cache is being invalidate because project 5e6cfc1041dd490fb95c476bd5fe0b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:13.900659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:13.900659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f3040631-1d4b-45d7-af9a-32cfe84507a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:13.900659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1005/2003] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:13 2022] DELETE /identity/v3/projects/5e6cfc1041dd490fb95c476bd5fe0b44 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:13.904457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:13.904886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:13.905006 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:13.928468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:13.928468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:13.997282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 999/2004] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:14.005841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.006159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.006407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.031581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:14.031581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:14.038407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1006/2005] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:14.042751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:14.053517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffa045-79e3-4271-97d8-6612e0b3e5a8 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:14.058437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:14.059060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.059347 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.059621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.060276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:14.060626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:14.062337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:14.078723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1000/2006] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:14.079783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:14.090162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a29bea-6c8b-411d-813c-bce318359234 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:14.095203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:14.095900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.097343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.097875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.098530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:14.100603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:14.197480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1007/2007] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:14 2022] POST /identity/v3/projects => generated 359 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:05:14.207481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:14.218016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b813e-8bfc-4066-87b9-853eec87d5a3 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:14.279647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:14.280264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.280583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.280858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.281494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:14.283955 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:14.398761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1001/2008] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:14 2022] POST /identity/v3/users => generated 280 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:14.400153 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:14.410900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04feca24-4002-46b2-ac22-119492f61de3 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:14.415444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:14.416047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.416363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.416639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.417479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:14.419051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:14.429543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1008/2009] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:14.434456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:14.444932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907bb3-ec75-4f4a-ae31-13dc15e4c1ac admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:14.449589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ed33c-e164-4c1c-8d60-d4fcd4b28162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:14.450255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ed33c-e164-4c1c-8d60-d4fcd4b28162 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.450538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ed33c-e164-4c1c-8d60-d4fcd4b28162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.450861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ed33c-e164-4c1c-8d60-d4fcd4b28162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.451461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ed33c-e164-4c1c-8d60-d4fcd4b28162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:14.453065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ed33c-e164-4c1c-8d60-d4fcd4b28162 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:14.463972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1002/2010] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:14.468853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:14.480391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44df2b-7d30-454a-a2bd-47941aec6e09 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:14.484968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:14.485622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.485907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.486182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] PATH_INFO: `/v3/projects/17eeee4b4db548808f317d4e15f8f563/users/a8bfabb881494665a92f8f6b08dc4438/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.505037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17eeee4b4db548808f317d4e15f8f563, user_id=a8bfabb881494665a92f8f6b08dc4438, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:14.507811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:14.584884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1009/2011] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:14 2022] PUT /identity/v3/projects/17eeee4b4db548808f317d4e15f8f563/users/a8bfabb881494665a92f8f6b08dc4438/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:14.595845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:14.596140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:14.596462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:14.632152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] MFA Rules not processed for user `a8bfabb881494665a92f8f6b08dc4438`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:14.634087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:14.653240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:14.719021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1003/2012] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:15.014574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:15.016392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:15.094665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04889d2-a453-43e9-b239-520b70a797f9 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:15.099316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af48a9-d700-4dd5-a477-9c96442af6ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:15.100035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af48a9-d700-4dd5-a477-9c96442af6ac service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:15.100474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af48a9-d700-4dd5-a477-9c96442af6ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:15.100834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af48a9-d700-4dd5-a477-9c96442af6ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:15.112233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af48a9-d700-4dd5-a477-9c96442af6ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:15.114889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af48a9-d700-4dd5-a477-9c96442af6ac service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:15.128514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1010/2013] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:15.161640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:15.274005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e1b8-5c26-46c4-b4cd-b305efe3c47d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:15.281302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:15.282522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:15.282898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:15.283265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] PATH_INFO: `/v3/users/a8bfabb881494665a92f8f6b08dc4438` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:15.285304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bfabb881494665a92f8f6b08dc4438)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:15.287971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:15.608644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bfabb881494665a92f8f6b08dc4438'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:15.626514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bfabb881494665a92f8f6b08dc4438'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:15.628443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bfabb881494665a92f8f6b08dc4438'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:15.629005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bfabb881494665a92f8f6b08dc4438'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:15.718686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bfabb881494665a92f8f6b08dc4438'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:15.748840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1004/2014] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:15 2022] DELETE /identity/v3/users/a8bfabb881494665a92f8f6b08dc4438 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:15.754321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:15.754604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:15.755210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:15.787427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:15.788710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:15.845232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:15.857012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1011/2015] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:15.868164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:15.869722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:15.914194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:15.933469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef5aa1-6091-4be1-a384-ce8ecc8c5242 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:15.937671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:15.938366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:15.938723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:15.939305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:15.959892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:15.961373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:15.972370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1005/2016] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:16.548406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:16.552012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:16.575949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7bc84-4148-4da9-b6de-4d3d55ca6b4f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:16.581435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:16.581993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:16.582279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:16.582557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] PATH_INFO: `/v3/projects/17eeee4b4db548808f317d4e15f8f563` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:16.584370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17eeee4b4db548808f317d4e15f8f563)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:16.592393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:16.850465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17eeee4b4db548808f317d4e15f8f563', 'request_id': 'req-5bf7c86e-266d-48db-8632-534ca5fb0a55'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:16.989198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17eeee4b4db548808f317d4e15f8f563', 'request_id': 'req-5bf7c86e-266d-48db-8632-534ca5fb0a55'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:16.995195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17eeee4b4db548808f317d4e15f8f563', 'request_id': 'req-5bf7c86e-266d-48db-8632-534ca5fb0a55'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:17.007483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] The token cache is being invalidate because project 17eeee4b4db548808f317d4e15f8f563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:17.007770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:17.008427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5bf7c86e-266d-48db-8632-534ca5fb0a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:17.009926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1012/2017] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:16 2022] DELETE /identity/v3/projects/17eeee4b4db548808f317d4e15f8f563 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:18.338793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:18.340575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:18.361251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c09310-65bb-4ec6-9af8-e00adeb7df42 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:18.435459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:18.436047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:18.436348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:18.436626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] PATH_INFO: `/v3/users/6e0708974466456696daf3e5cef3498e` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:18.437744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0708974466456696daf3e5cef3498e)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:18.440397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:18.820558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0708974466456696daf3e5cef3498e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:18.827342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0708974466456696daf3e5cef3498e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:18.828036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0708974466456696daf3e5cef3498e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:18.828675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0708974466456696daf3e5cef3498e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:19.006068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0708974466456696daf3e5cef3498e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:19.025834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1006/2018] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:18 2022] DELETE /identity/v3/users/6e0708974466456696daf3e5cef3498e => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:19.030542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:19.030995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:19.031106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:19.050829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:19.052039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:19.120946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1013/2019] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:19.134201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:19.135590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:19.202557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391b115-ac6d-44ce-a18f-ca2bc55f614c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:19.206260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:19.206874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:19.207151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:19.207515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:19.216787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:19.218275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:19.223768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:19.231598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1007/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:19.845602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:19.847857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:19.863160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac6ed-bb42-45cb-a928-bad3fdfb85c5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:19.868124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:19.868820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:19.869096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:19.869362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] PATH_INFO: `/v3/projects/b5b31cc9493d4bf69d403348cdb319ac` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:19.870506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b31cc9493d4bf69d403348cdb319ac)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:19.872441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:20.061572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b31cc9493d4bf69d403348cdb319ac', 'request_id': 'req-fcc337e1-b7df-4017-a458-911d9a9ebc5c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.171908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b31cc9493d4bf69d403348cdb319ac', 'request_id': 'req-fcc337e1-b7df-4017-a458-911d9a9ebc5c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.179920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b31cc9493d4bf69d403348cdb319ac', 'request_id': 'req-fcc337e1-b7df-4017-a458-911d9a9ebc5c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.197509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] The token cache is being invalidate because project b5b31cc9493d4bf69d403348cdb319ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:20.197815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.198616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.201127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1014/2021] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:19 2022] DELETE /identity/v3/projects/b5b31cc9493d4bf69d403348cdb319ac => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:20.206706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:20.218156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd754-83cc-495d-bd50-413872c138bb service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:20.298102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:20.298739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:20.299557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:20.299831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] PATH_INFO: `/v3/users/599056493c034b6a839b0620a60042f6` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:20.301064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=599056493c034b6a839b0620a60042f6)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:20.302963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:20.530103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '599056493c034b6a839b0620a60042f6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.538373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '599056493c034b6a839b0620a60042f6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.539171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '599056493c034b6a839b0620a60042f6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.539923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '599056493c034b6a839b0620a60042f6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.614918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a52b22a0-95fc-45c7-b50a-de36c8bcabb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '599056493c034b6a839b0620a60042f6'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:20.633490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1008/2022] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:20 2022] DELETE /identity/v3/users/599056493c034b6a839b0620a60042f6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:21.119743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:21.121208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:21.134868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc337e1-b7df-4017-a458-911d9a9ebc5c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:21.138749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:21.193062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:21.193617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.193896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.194168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] PATH_INFO: `/v3/projects/92f6d860a0904e948df159faf010de58` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.195272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f6d860a0904e948df159faf010de58)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:21.197192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:21.315076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f6d860a0904e948df159faf010de58', 'request_id': 'req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:21.389223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f6d860a0904e948df159faf010de58', 'request_id': 'req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:21.396183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f6d860a0904e948df159faf010de58', 'request_id': 'req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:21.409595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] The token cache is being invalidate because project 92f6d860a0904e948df159faf010de58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:21.409929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:21.410547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-41dc4b48-7a96-4a8b-b31f-69280b5d5c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:21.412148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1015/2023] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:21 2022] DELETE /identity/v3/projects/92f6d860a0904e948df159faf010de58 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:21.418820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.419295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.419419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.440986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:21.442191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:21.506415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1009/2024] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:21.514060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.514348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.514632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.534807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:21.536021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:21.544047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1016/2025] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:21.548254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:21.557968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea756eaf-a225-45cd-9192-0887d6adaebb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:21.562461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:21.563055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.563333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.563602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.564247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:21.565135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:21.566723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:21.578061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1010/2026] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:21.582808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:21.591971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b42bc-858f-4cee-ae1a-5f78ed5b27ca None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:21.596715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:21.597306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.597586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.597860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.598497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:21.600507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:21.697310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1017/2027] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:21 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:21.701494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:21.712784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0da667-56dc-49e3-91ab-e2ef417d7911 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:21.716642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:21.771692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:21.772339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.772729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.773012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.773659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:21.775713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:21.936221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1011/2028] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:21 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:21.940963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:21.950887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f0b7a-9411-49e0-a41d-22a3e686f8b2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:21.955588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:21.956188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.956505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.956777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.957419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:21.958999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:21.968721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1018/2029] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:21.972689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:21.983156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835699-9eec-4e83-ba20-34e7c791391f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:21.988664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bd545-503f-41c2-9fe3-a3643d886648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:21.989243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bd545-503f-41c2-9fe3-a3643d886648 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:21.989524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bd545-503f-41c2-9fe3-a3643d886648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:21.989803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bd545-503f-41c2-9fe3-a3643d886648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:21.990626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bd545-503f-41c2-9fe3-a3643d886648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:21.992219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bd545-503f-41c2-9fe3-a3643d886648 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.004053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1012/2030] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:22.008120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.017958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d71b4-8de8-4f3a-817f-676dd7ceac73 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.022394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.023029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.023324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.023578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] PATH_INFO: `/v3/projects/93c0f00ed5bb4744b4f2a950486763ea/users/d1330acfc0c74098a55d130f097509e4/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.037954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c0f00ed5bb4744b4f2a950486763ea, user_id=d1330acfc0c74098a55d130f097509e4, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.040618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.115222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1019/2031] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:22 2022] PUT /identity/v3/projects/93c0f00ed5bb4744b4f2a950486763ea/users/d1330acfc0c74098a55d130f097509e4/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:22.125382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.125671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.125949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.153657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] MFA Rules not processed for user `d1330acfc0c74098a55d130f097509e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:22.154866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:22.218325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1013/2032] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:22.223499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.260591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:22.287725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fa0a5-12a2-43c8-8cca-8522af37d9e7 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.292934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.293540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.293822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.294097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.294740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.296669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.406849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1020/2033] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:22 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:22.411667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.422623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c69f74-b8b3-4f22-bcf5-53272cdb02f5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.479449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.480095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.480405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.480681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.481312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.483275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.602527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1014/2034] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:22 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:22.606684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.621058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b45a6-a202-449b-83dc-38433fd6d87d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.623214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.623827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.624111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.624421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.625076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.626746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.637422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1021/2035] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:22.641776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.648813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:22.652707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8b9b-d513-4ceb-8cde-94f1f6f5d121 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.660443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.660443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.660443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.660443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] PATH_INFO: `/v3/projects/51e31f64a1b34fd4880a78ddd6a3857b/users/ebcb99525b244102b5cb9e042220fb60/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.675358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e31f64a1b34fd4880a78ddd6a3857b, user_id=ebcb99525b244102b5cb9e042220fb60, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.678038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.752912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1015/2036] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:22 2022] PUT /identity/v3/projects/51e31f64a1b34fd4880a78ddd6a3857b/users/ebcb99525b244102b5cb9e042220fb60/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:22.756881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.767129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d963d7d-e4e2-445d-bb84-b3904d6b6514 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.832591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b265dda-d8fe-4ddb-8aef-6d1b38405e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.833206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b265dda-d8fe-4ddb-8aef-6d1b38405e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.833490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b265dda-d8fe-4ddb-8aef-6d1b38405e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.834090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b265dda-d8fe-4ddb-8aef-6d1b38405e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.834744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b265dda-d8fe-4ddb-8aef-6d1b38405e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.836403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b265dda-d8fe-4ddb-8aef-6d1b38405e17 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.846870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1022/2037] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:05:22.851316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:22.861694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d61d3-b6b0-4379-9352-8c5f865fa4f3 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:22.867882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:22.868846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.869271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.869674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] PATH_INFO: `/v3/projects/51e31f64a1b34fd4880a78ddd6a3857b/users/ebcb99525b244102b5cb9e042220fb60/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.872423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e31f64a1b34fd4880a78ddd6a3857b, user_id=ebcb99525b244102b5cb9e042220fb60, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:22.876627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:22.892127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1016/2038] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:22 2022] PUT /identity/v3/projects/51e31f64a1b34fd4880a78ddd6a3857b/users/ebcb99525b244102b5cb9e042220fb60/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:22.904471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:22.904763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:22.905046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:22.934237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] MFA Rules not processed for user `ebcb99525b244102b5cb9e042220fb60`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:22.935434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:23.003748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1023/2039] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:23.014366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:23.016100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:23.072805 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33a9d5-d244-4557-97e8-e8b2f9c68763 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:23.076423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9557bfa-7d49-4efa-ba90-1f29945a709a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:23.077019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9557bfa-7d49-4efa-ba90-1f29945a709a service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:23.077296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9557bfa-7d49-4efa-ba90-1f29945a709a service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:23.077562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9557bfa-7d49-4efa-ba90-1f29945a709a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:23.086995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9557bfa-7d49-4efa-ba90-1f29945a709a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:23.088430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9557bfa-7d49-4efa-ba90-1f29945a709a service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:23.097791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1017/2040] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:24.593089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:24.625605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:24.656892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902eea-7e4c-4477-8b19-049fd79abdd6 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:24.661890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:24.662522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:24.662800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:24.663072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] PATH_INFO: `/v3/users/d1330acfc0c74098a55d130f097509e4` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:24.664197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1330acfc0c74098a55d130f097509e4)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:24.666185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:24.997893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1330acfc0c74098a55d130f097509e4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:25.004916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1330acfc0c74098a55d130f097509e4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:25.148903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1330acfc0c74098a55d130f097509e4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:25.149537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1330acfc0c74098a55d130f097509e4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:25.162832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1330acfc0c74098a55d130f097509e4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:25.168896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1024/2041] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:24 2022] DELETE /identity/v3/users/d1330acfc0c74098a55d130f097509e4 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:25.173399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:25.174497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:25.174615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:25.193965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:25.196935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:25.254307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1018/2042] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:25.263917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:25.265366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:25.322608 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6bfd5-e6a1-4a0f-8cf5-4eb15f77e232 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:25.326364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:25.326981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:25.327266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:25.327572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:25.336914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:25.338368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:25.359408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1025/2043] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:25.982369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:25.983962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:25.996251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d398be-2218-4de6-986b-6d187488f8a0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:26.001054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:26.001733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:26.002578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:26.002864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] PATH_INFO: `/v3/projects/93c0f00ed5bb4744b4f2a950486763ea` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:26.003975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c0f00ed5bb4744b4f2a950486763ea)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:26.005909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:26.224239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c0f00ed5bb4744b4f2a950486763ea', 'request_id': 'req-a4abe997-d58c-46e7-947b-88ebee83a2fc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:26.387085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c0f00ed5bb4744b4f2a950486763ea', 'request_id': 'req-a4abe997-d58c-46e7-947b-88ebee83a2fc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:26.393332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c0f00ed5bb4744b4f2a950486763ea', 'request_id': 'req-a4abe997-d58c-46e7-947b-88ebee83a2fc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:26.398174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:26.405836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] The token cache is being invalidate because project 93c0f00ed5bb4744b4f2a950486763ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:26.406035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:26.406701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:26.408603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1019/2044] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:25 2022] DELETE /identity/v3/projects/93c0f00ed5bb4744b4f2a950486763ea => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:26.412510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:26.422078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bad04-be53-418e-baa4-db3344c65d0f service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:26.472933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:26.482409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:26.483041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:26.483326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:26.483597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] PATH_INFO: `/v3/users/ebcb99525b244102b5cb9e042220fb60` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:26.484755 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcb99525b244102b5cb9e042220fb60)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:26.487432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:26.892927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcb99525b244102b5cb9e042220fb60'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:26.901128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcb99525b244102b5cb9e042220fb60'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:27.075969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcb99525b244102b5cb9e042220fb60'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:27.076711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcb99525b244102b5cb9e042220fb60'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:27.090219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3dfe5904-b7bd-4acf-8191-788e64809215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcb99525b244102b5cb9e042220fb60'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:27.097001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1026/2045] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:26 2022] DELETE /identity/v3/users/ebcb99525b244102b5cb9e042220fb60 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:27.741114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:27.741114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:27.750126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abe997-d58c-46e7-947b-88ebee83a2fc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:27.847597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:27.848288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:27.848606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:27.848879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] PATH_INFO: `/v3/projects/51e31f64a1b34fd4880a78ddd6a3857b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:27.850071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e31f64a1b34fd4880a78ddd6a3857b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:27.851984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.101119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e31f64a1b34fd4880a78ddd6a3857b', 'request_id': 'req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:28.183102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e31f64a1b34fd4880a78ddd6a3857b', 'request_id': 'req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:28.189242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e31f64a1b34fd4880a78ddd6a3857b', 'request_id': 'req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:28.200894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] The token cache is being invalidate because project 51e31f64a1b34fd4880a78ddd6a3857b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:28.201180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:28.201897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0707150a-88cb-4ab5-9ae2-c86877b2a3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:28.203478 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1020/2046] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:27 2022] DELETE /identity/v3/projects/51e31f64a1b34fd4880a78ddd6a3857b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:28.208551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.209057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.209161 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.230115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:28.231319 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:28.293927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1027/2047] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:28.302483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.302810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.303090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.322499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:28.323711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:28.331938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1021/2048] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:28.336137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:28.346509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d80078-3c6a-46ef-b6d5-436b128c1d51 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:28.362619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:28.362619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.362619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.362619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.363566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:28.363566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:28.363566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.372290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1028/2049] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:28.373542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:28.384578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8009b-606b-475c-a7a9-19c9f96aaf3c None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:28.390972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:28.391987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.392177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.392486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.393159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:28.395062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.493792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1022/2050] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:28 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:28.497924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:28.511031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059335-d303-4b22-b11d-3e757168570e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:28.581543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:28.582275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.582626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.582976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.583700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:28.585834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.595722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:28.711549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1029/2051] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:28 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:05:28.715640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:28.731198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739cb6e-90a4-46de-9048-1dd55d36df6e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:28.733491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:28.734062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.734577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.735214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.735541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:28.737418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.749806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1023/2052] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:28.754267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:28.765332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081b542-7e56-4b15-862b-26ea42c8ffbd admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:28.773046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3017127-13db-4af0-a2cd-c10fb5148876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:28.773665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3017127-13db-4af0-a2cd-c10fb5148876 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.773945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3017127-13db-4af0-a2cd-c10fb5148876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.774218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3017127-13db-4af0-a2cd-c10fb5148876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.774880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3017127-13db-4af0-a2cd-c10fb5148876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:28.776538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3017127-13db-4af0-a2cd-c10fb5148876 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.790260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1030/2053] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:05:28.794629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:28.826528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5ad05-4919-440f-965c-c4c0771ee807 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:28.826528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:28.826528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.826528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.826884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] PATH_INFO: `/v3/projects/9b16b1984a3047a3b589fe341bcf836c/users/4e0e24b4148d4f6e95f4c65570c79e94/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:28.833574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b16b1984a3047a3b589fe341bcf836c, user_id=4e0e24b4148d4f6e95f4c65570c79e94, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:28.836613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:28.944303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1024/2054] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:28 2022] PUT /identity/v3/projects/9b16b1984a3047a3b589fe341bcf836c/users/4e0e24b4148d4f6e95f4c65570c79e94/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:28.957841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:28.957841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:28.957841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:29.006659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] MFA Rules not processed for user `4e0e24b4148d4f6e95f4c65570c79e94`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:29.008031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:29.092700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1031/2055] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:29.105004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:29.107493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:29.142148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:29.197387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca832a8a-910c-405a-bb67-d9cb2e73ad6d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:29.201964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfe16f-077d-49ee-ab58-937b819e6449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:29.202695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe16f-077d-49ee-ab58-937b819e6449 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:29.203276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe16f-077d-49ee-ab58-937b819e6449 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:29.203762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfe16f-077d-49ee-ab58-937b819e6449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:29.218693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfe16f-077d-49ee-ab58-937b819e6449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:29.220807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfe16f-077d-49ee-ab58-937b819e6449 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:29.233190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1025/2056] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:29.276519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:29.367884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f6c5f-cf11-4604-a312-ed4898768f52 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:29.373327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:29.374224 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:29.374658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:29.375087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] PATH_INFO: `/v3/users/4e0e24b4148d4f6e95f4c65570c79e94` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:29.376570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0e24b4148d4f6e95f4c65570c79e94)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:29.380414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:29.722459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0e24b4148d4f6e95f4c65570c79e94'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:29.733600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0e24b4148d4f6e95f4c65570c79e94'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:29.759791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20741536-34ce-4d30-ae53-25cf96b18a26 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:29.760332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20741536-34ce-4d30-ae53-25cf96b18a26 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:29.760978 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20741536-34ce-4d30-ae53-25cf96b18a26 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:29.762590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1026/2057] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 18:05:29.812116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:29.812693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:29.812968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:29.821316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0e24b4148d4f6e95f4c65570c79e94'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:29.824524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0e24b4148d4f6e95f4c65570c79e94'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:29.837582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:29.839122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:29.842031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1411d127-bc71-46e0-a1b1-2e6a781ba67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0e24b4148d4f6e95f4c65570c79e94'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:29.855789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1032/2058] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:29 2022] DELETE /identity/v3/users/4e0e24b4148d4f6e95f4c65570c79e94 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:29.860592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:29.864722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:29.864722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:29.884267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:29.885501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:29.913882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1027/2059] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:05:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:29.939075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:29.939075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:29.959911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1033/2060] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:29.961086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf8819-8936-4497-b5f1-52db4096a7b6 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:29.969317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:29.971582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:30.047593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8602c-ac86-4a2d-98ea-c7ba23bd846e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:30.052740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f2b5-859c-43f5-acb5-3ea171ce7f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:30.053392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f2b5-859c-43f5-acb5-3ea171ce7f7a service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:30.053779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f2b5-859c-43f5-acb5-3ea171ce7f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:30.054198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f2b5-859c-43f5-acb5-3ea171ce7f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:30.072541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f2b5-859c-43f5-acb5-3ea171ce7f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:30.072541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f2b5-859c-43f5-acb5-3ea171ce7f7a service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:30.078847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3a5f2-8ba8-45f1-b6f1-d74f05b27f0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:30.079675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3a5f2-8ba8-45f1-b6f1-d74f05b27f0d service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:30.080697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3a5f2-8ba8-45f1-b6f1-d74f05b27f0d service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:30.080990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3a5f2-8ba8-45f1-b6f1-d74f05b27f0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:30.089056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1034/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:30.111654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3a5f2-8ba8-45f1-b6f1-d74f05b27f0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:30.114764 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3a5f2-8ba8-45f1-b6f1-d74f05b27f0d service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:30.147537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1028/2062] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:30.327426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:30.863124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69245-ab96-4ea8-b4e9-85aa74a4406b None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:30.863411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69245-ab96-4ea8-b4e9-85aa74a4406b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:30.863687 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69245-ab96-4ea8-b4e9-85aa74a4406b None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:30.865329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1035/2063] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 18:05:30.872965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:30.874481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:30.875438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:30.884485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d69245-ab96-4ea8-b4e9-85aa74a4406b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:30.887426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-51d69245-ab96-4ea8-b4e9-85aa74a4406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:30.905333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:30.906979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:30.917039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1029/2064] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:05:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:30.918653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d69245-ab96-4ea8-b4e9-85aa74a4406b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:30.928953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:30.930593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:30.931310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:30.931406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:30.931976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] PATH_INFO: `/v3/users/1e2be61ea35a47269c0454f023bbbf20` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:30.935282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2be61ea35a47269c0454f023bbbf20)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:30.941604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:30.943455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4ddd5-d2cd-42ea-878b-2e4f6c416b4b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:30.951545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:30.951545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:30.951545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:30.951545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:30.963921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:30.965671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:30.979337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1030/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:31.154751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:31.157916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.175948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096fe87-18f5-475d-b3fc-60a4b9710881 service placement] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:31.187992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:31.188689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.188962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.189454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] PATH_INFO: `/v3/users/d87d9ed756014d73a8bc326221e97e94` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.190893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87d9ed756014d73a8bc326221e97e94)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:31.192871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:31.223046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2be61ea35a47269c0454f023bbbf20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.234171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:31.241560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2be61ea35a47269c0454f023bbbf20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.333356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2be61ea35a47269c0454f023bbbf20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.334110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2be61ea35a47269c0454f023bbbf20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.348794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2be61ea35a47269c0454f023bbbf20'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.355425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1036/2066] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:30 2022] DELETE /identity/v3/users/1e2be61ea35a47269c0454f023bbbf20 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:31.356648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:31.360512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.436663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b1847-4dd2-44c7-8a6a-445a9cf94f55 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:31.440989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87d9ed756014d73a8bc326221e97e94'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.442093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:31.442734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.443010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.443309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] PATH_INFO: `/v3/projects/9b16b1984a3047a3b589fe341bcf836c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.444469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b16b1984a3047a3b589fe341bcf836c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:31.446385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:31.449535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87d9ed756014d73a8bc326221e97e94'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.450409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87d9ed756014d73a8bc326221e97e94'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.451243 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87d9ed756014d73a8bc326221e97e94'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.566917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a76d07e3-61e7-4325-8046-b7b546c837d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87d9ed756014d73a8bc326221e97e94'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.599419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1031/2067] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:31 2022] DELETE /identity/v3/users/d87d9ed756014d73a8bc326221e97e94 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:31.603220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.603512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.603784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.607202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b16b1984a3047a3b589fe341bcf836c', 'request_id': 'req-5749414a-4977-4f63-b95a-c44de894ca0c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.641989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:31.643882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.729437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b16b1984a3047a3b589fe341bcf836c', 'request_id': 'req-5749414a-4977-4f63-b95a-c44de894ca0c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.729752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1032/2068] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:31.739512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b16b1984a3047a3b589fe341bcf836c', 'request_id': 'req-5749414a-4977-4f63-b95a-c44de894ca0c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.740121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:31.741689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.759945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] The token cache is being invalidate because project 9b16b1984a3047a3b589fe341bcf836c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:31.760248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.761154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5749414a-4977-4f63-b95a-c44de894ca0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:31.764234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1037/2069] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:31 2022] DELETE /identity/v3/projects/9b16b1984a3047a3b589fe341bcf836c => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:31.778489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa91ee-db8a-4225-b6ed-dc9c170401f7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.779302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa91ee-db8a-4225-b6ed-dc9c170401f7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.779820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa91ee-db8a-4225-b6ed-dc9c170401f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.807300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-90fa91ee-db8a-4225-b6ed-dc9c170401f7 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:31.808741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-90fa91ee-db8a-4225-b6ed-dc9c170401f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.819732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1038/2070] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:31.829036 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.829334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.829614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.849062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0f32-c7f6-49c1-886e-c959f81e1bbb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:31.854285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:31.855029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.855506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:31.855707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.856009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.856760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.858451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.868170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1039/2071] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:31.872083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:31.878777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:31.880213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:31.883995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7d29-7dcb-40e3-8611-fd290c2f961f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:31.890716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:31.891326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.891761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.892186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.893450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1033/2072] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:31.893752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:31.894062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:31.895708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:31.908697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1040/2073] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:05:31.913449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:31.923013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48611b89-7d3e-49c5-b9aa-b5e92edf23f2 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:31.927853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:31.928560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.928901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.929338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.929981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:31.931943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:31.950381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:31.952187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:31.975729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbae637-dd25-4310-b046-177a566b2b9c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:31.980689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:31.981445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:31.981796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:31.982080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] PATH_INFO: `/v3/users/af540344a27b48eba16db2861df2078b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:31.983236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=af540344a27b48eba16db2861df2078b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:31.985235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:32.027266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1034/2074] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:31 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:32.027266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:32.028841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:32.092422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83908c-9d1d-4fd8-84ba-eb3b7853a730 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:32.097449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:32.098085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:32.098367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:32.098639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] PATH_INFO: `/v3/projects/99fad5e0b9aa4e3a8a69bcc270caf99b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:32.099784 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fad5e0b9aa4e3a8a69bcc270caf99b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:32.101704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:32.252585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af540344a27b48eba16db2861df2078b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.260807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fad5e0b9aa4e3a8a69bcc270caf99b', 'request_id': 'req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.262504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af540344a27b48eba16db2861df2078b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.306479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:32.338115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fad5e0b9aa4e3a8a69bcc270caf99b', 'request_id': 'req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.347864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fad5e0b9aa4e3a8a69bcc270caf99b', 'request_id': 'req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.373419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] The token cache is being invalidate because project 99fad5e0b9aa4e3a8a69bcc270caf99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:32.373892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.374502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.385421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1035/2075] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:32 2022] DELETE /identity/v3/projects/99fad5e0b9aa4e3a8a69bcc270caf99b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:32.385421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:32.385421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:32.403786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af540344a27b48eba16db2861df2078b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.404691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af540344a27b48eba16db2861df2078b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.419685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af540344a27b48eba16db2861df2078b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.441521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1041/2076] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:31 2022] DELETE /identity/v3/users/af540344a27b48eba16db2861df2078b => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:32.442306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:32.446679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:32.492664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468e3e3-4f2b-4efd-b851-c4f38606e9e2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:32.552922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:32.563805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c25eb-6c89-4e5c-aa78-2cd7d0a5c5ef admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:32.569787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:32.570537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:32.571034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:32.571349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] PATH_INFO: `/v3/projects/ab7e06dcbf3442ca98787f5201ddad64` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:32.572607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7e06dcbf3442ca98787f5201ddad64)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:32.574667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:32.584965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:32.585617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:32.585893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:32.586159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:32.586916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:32.588908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:32.735889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1036/2077] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:32 2022] POST /identity/v3/users => generated 287 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:32.735889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:32.737855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:32.745420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7e06dcbf3442ca98787f5201ddad64', 'request_id': 'req-05adcb75-5507-4390-96e4-f927e951e44c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.749780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070ec8-ff8a-46fd-9741-12d9151e9bd0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:32.815670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:32.816408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:32.816808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:32.817188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] PATH_INFO: `/v3/users/2708c116cb1f4631bf0722e1145ce23e` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:32.818458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2708c116cb1f4631bf0722e1145ce23e)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:32.820571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:32.883389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7e06dcbf3442ca98787f5201ddad64', 'request_id': 'req-05adcb75-5507-4390-96e4-f927e951e44c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.889433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7e06dcbf3442ca98787f5201ddad64', 'request_id': 'req-05adcb75-5507-4390-96e4-f927e951e44c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.900304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] The token cache is being invalidate because project ab7e06dcbf3442ca98787f5201ddad64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:32.900617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.901277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:32.902778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1042/2078] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:32 2022] DELETE /identity/v3/projects/ab7e06dcbf3442ca98787f5201ddad64 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:32.903545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:32.904853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:32.917983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adcb75-5507-4390-96e4-f927e951e44c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:32.922581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:32.923157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:32.923436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:32.923916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:32.925125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:32.926713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:32.936421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1043/2079] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:32.937191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:32.938749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:32.951212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e8187-efbc-473b-a09f-fb12a4a77495 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:32.968674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:32.968674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:32.968674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:32.968674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] PATH_INFO: `/v3/projects/62c83a25c5f74e3396ebe8b0a3ee580e` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:32.969123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c83a25c5f74e3396ebe8b0a3ee580e)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:32.969123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:33.434289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2708c116cb1f4631bf0722e1145ce23e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.434867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c83a25c5f74e3396ebe8b0a3ee580e', 'request_id': 'req-dba9170e-e064-4e20-8c15-3b6338bd190d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.441845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2708c116cb1f4631bf0722e1145ce23e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.442522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2708c116cb1f4631bf0722e1145ce23e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.443130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2708c116cb1f4631bf0722e1145ce23e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.583169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c83a25c5f74e3396ebe8b0a3ee580e', 'request_id': 'req-dba9170e-e064-4e20-8c15-3b6338bd190d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.590302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c83a25c5f74e3396ebe8b0a3ee580e', 'request_id': 'req-dba9170e-e064-4e20-8c15-3b6338bd190d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.605871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] The token cache is being invalidate because project 62c83a25c5f74e3396ebe8b0a3ee580e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:33.605871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.605871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.607787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1044/2080] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:32 2022] DELETE /identity/v3/projects/62c83a25c5f74e3396ebe8b0a3ee580e => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:33.608668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:33.610079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:33.675591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2708c116cb1f4631bf0722e1145ce23e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.683400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9170e-e064-4e20-8c15-3b6338bd190d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:33.690731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:33.691377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:33.691653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:33.691923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] PATH_INFO: `/v3/users/76c61940b7ee42a1b927bf768eac4473` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:33.693165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c61940b7ee42a1b927bf768eac4473)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:33.695128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:33.697312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1037/2081] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:32 2022] DELETE /identity/v3/users/2708c116cb1f4631bf0722e1145ce23e => generated 0 bytes in 964 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:33.698328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:33.701097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:33.779576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82745a-4cb9-4699-bca6-ea533e69c4d8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:33.789377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60234e9c-410b-452e-8ba5-27ec0c4e2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:33.790087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60234e9c-410b-452e-8ba5-27ec0c4e2e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:33.790649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60234e9c-410b-452e-8ba5-27ec0c4e2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:33.791056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60234e9c-410b-452e-8ba5-27ec0c4e2e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:33.792451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60234e9c-410b-452e-8ba5-27ec0c4e2e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:33.794401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60234e9c-410b-452e-8ba5-27ec0c4e2e42 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:33.811857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1038/2082] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:33.813293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:33.813520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:33.813789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:33.834980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:33.836418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:33.845612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1039/2083] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:33.846862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:33.849527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:33.862753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9c883-621a-4ce8-b87d-645bd597443d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:33.867360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec23918-e44f-408c-894c-5ad5e96f4a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:33.868057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec23918-e44f-408c-894c-5ad5e96f4a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:33.868363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec23918-e44f-408c-894c-5ad5e96f4a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:33.868645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec23918-e44f-408c-894c-5ad5e96f4a20 admin admin] PATH_INFO: `/v3/projects/7045cbcdfcda411ba59d9e930410d6e0/users/e6207262d90148d3b478208e92ec8d1f/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:33.887102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec23918-e44f-408c-894c-5ad5e96f4a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7045cbcdfcda411ba59d9e930410d6e0, user_id=e6207262d90148d3b478208e92ec8d1f, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:33.889875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec23918-e44f-408c-894c-5ad5e96f4a20 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:33.929016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c61940b7ee42a1b927bf768eac4473'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:33.935869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c61940b7ee42a1b927bf768eac4473'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.083984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1040/2084] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:33 2022] PUT /identity/v3/projects/7045cbcdfcda411ba59d9e930410d6e0/users/e6207262d90148d3b478208e92ec8d1f/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:34.085274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.085558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.085886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.104605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:34.105806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.114954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c61940b7ee42a1b927bf768eac4473'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.115641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c61940b7ee42a1b927bf768eac4473'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.135260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-3341a112-b08c-4124-b698-9cd7d58bb8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c61940b7ee42a1b927bf768eac4473'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.141948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1045/2085] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:33 2022] DELETE /identity/v3/users/76c61940b7ee42a1b927bf768eac4473 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:34.143694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.143969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.144506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.178197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] MFA Rules not processed for user `e6207262d90148d3b478208e92ec8d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:34.179570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.183886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1041/2086] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:34.184845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:34.187531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.198787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b389ff-ebfe-404e-9ebf-f2c6b3168730 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:34.261877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1046/2087] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:34.262514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:34.271064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:34.271944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.272131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.272503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] PATH_INFO: `/v3/projects/7fea72bddc6044599a875528ea4eaec7` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.274288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fea72bddc6044599a875528ea4eaec7)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:34.276520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:34.324675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76f90-13cf-4f54-9a7f-fdcdda9a0f42 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:34.329937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:34.330553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.330890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.331160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.331805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:34.332108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:34.333739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:34.344766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1047/2088] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:34.346117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:34.347820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.407295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0a858-956e-4dcd-bb10-b22725910d4a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:34.413703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:34.414393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.414748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.415142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.425981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:34.427594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:34.435230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fea72bddc6044599a875528ea4eaec7', 'request_id': 'req-412c7556-c0e9-423c-adac-75c0c5b4c09b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.454111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1048/2089] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:34.454111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:34.454111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fd4d8-4c25-4b77-b9c3-02d6f34c5555 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:34.515096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231ea60-2ff4-47df-a60e-4f8890bea08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:34.515891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231ea60-2ff4-47df-a60e-4f8890bea08a admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.516302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231ea60-2ff4-47df-a60e-4f8890bea08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.516699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231ea60-2ff4-47df-a60e-4f8890bea08a admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.518037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231ea60-2ff4-47df-a60e-4f8890bea08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:34.520480 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231ea60-2ff4-47df-a60e-4f8890bea08a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:34.606105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fea72bddc6044599a875528ea4eaec7', 'request_id': 'req-412c7556-c0e9-423c-adac-75c0c5b4c09b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.613377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fea72bddc6044599a875528ea4eaec7', 'request_id': 'req-412c7556-c0e9-423c-adac-75c0c5b4c09b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.624437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] The token cache is being invalidate because project 7fea72bddc6044599a875528ea4eaec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:34.624729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.625429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.627561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1042/2090] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:34 2022] DELETE /identity/v3/projects/7fea72bddc6044599a875528ea4eaec7 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:34.628416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:34.629660 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.641775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7556-c0e9-423c-adac-75c0c5b4c09b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:34.646527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:34.647133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.649400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.649400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] PATH_INFO: `/v3/users/e6207262d90148d3b478208e92ec8d1f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.649400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6207262d90148d3b478208e92ec8d1f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:34.650721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:34.706956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1049/2091] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:34 2022] POST /identity/v3/projects => generated 403 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:34.707321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.707607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.707886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.737344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:34.738583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.795998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1050/2092] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:34.796871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:34.798104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:34.810615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae00b68-6cc7-49fc-a382-c08b4a66c331 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:34.816078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:34.816987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:34.817263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:34.817531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] PATH_INFO: `/v3/projects/23f17661792e458a8a6078130961618c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:34.818653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f17661792e458a8a6078130961618c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:34.820595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:34.871583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6207262d90148d3b478208e92ec8d1f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.879324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6207262d90148d3b478208e92ec8d1f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.880028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6207262d90148d3b478208e92ec8d1f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.880662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6207262d90148d3b478208e92ec8d1f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:34.980782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f17661792e458a8a6078130961618c', 'request_id': 'req-7697025d-0b90-4e88-b9f6-5c455439d561'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:35.051960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6207262d90148d3b478208e92ec8d1f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:35.072985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1043/2093] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:34 2022] DELETE /identity/v3/users/e6207262d90148d3b478208e92ec8d1f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:35.073951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.075277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.137977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b95f10-6288-4ddf-a7a3-b2d1442659d0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.143207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21022eec-f4ea-4b3b-939b-4697d11b34e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.143800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21022eec-f4ea-4b3b-939b-4697d11b34e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.144077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21022eec-f4ea-4b3b-939b-4697d11b34e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.144376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21022eec-f4ea-4b3b-939b-4697d11b34e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.145001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21022eec-f4ea-4b3b-939b-4697d11b34e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.146914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21022eec-f4ea-4b3b-939b-4697d11b34e6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.217669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f17661792e458a8a6078130961618c', 'request_id': 'req-7697025d-0b90-4e88-b9f6-5c455439d561'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:35.228959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f17661792e458a8a6078130961618c', 'request_id': 'req-7697025d-0b90-4e88-b9f6-5c455439d561'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:35.245767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] The token cache is being invalidate because project 23f17661792e458a8a6078130961618c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:35.246060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:35.246711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7697025d-0b90-4e88-b9f6-5c455439d561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:35.248733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1051/2094] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:34 2022] DELETE /identity/v3/projects/23f17661792e458a8a6078130961618c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:35.256477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33bdab-e57c-4473-8b69-2d82105a3bf6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.256750 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33bdab-e57c-4473-8b69-2d82105a3bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.262327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33bdab-e57c-4473-8b69-2d82105a3bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.279115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2f33bdab-e57c-4473-8b69-2d82105a3bf6 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:35.280305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2f33bdab-e57c-4473-8b69-2d82105a3bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.289618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1052/2095] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:35.290978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.291254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.291522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.293636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1044/2096] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/users => generated 302 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:35.293803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.296918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.296918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.314528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:35.315740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.321763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:35.322929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.323546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1053/2097] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:35.324341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.334985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0217e5-c837-4228-b039-bb0a6b387503 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.338437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1045/2098] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:35.339274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.340110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.340546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.340756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.341035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.341307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.341973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:35.342366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.343944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.352921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf9db5-f804-4b24-8548-ff75998424fd None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.359960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1054/2099] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:35.359960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.360166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256b53-b17c-4d78-ba97-08c869fbe6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.360166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256b53-b17c-4d78-ba97-08c869fbe6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.360166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256b53-b17c-4d78-ba97-08c869fbe6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.360166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256b53-b17c-4d78-ba97-08c869fbe6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.360556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256b53-b17c-4d78-ba97-08c869fbe6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.361168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256b53-b17c-4d78-ba97-08c869fbe6ba admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.361278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:35.362619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.374413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1046/2100] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:35.374413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.374413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.374413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.393225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:35.394429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:35.402400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1047/2101] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:35.403308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.413258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298a68c-63e1-4dd6-81ae-e07797ad5afb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.417818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.418403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.418677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.418948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.419569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.421470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.423343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b79d62-25f4-4a9c-8d70-780bc43187de admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.427115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.427735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.428012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.428371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.439150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.440682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.450234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1055/2102] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:35.451021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.460523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63656-90e6-4b9a-a90c-b13a3c145e24 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.465637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.466267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.466577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.466846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.467534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.469217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.480505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1056/2103] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:35.485570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.491347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0737c6-04aa-44bb-a55d-7fdfb2bec6ab admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.496263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.496891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.497174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.497451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.498105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:35.498408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.499994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.513337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1057/2104] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:35.514245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1048/2105] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:35.514996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.521197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.526500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17588e-ba96-4bb8-a005-8301abe4c6ef admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.585734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e09bc-1faa-4e30-a900-badddc0d5dc7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.588248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.588985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.589274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.589552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] PATH_INFO: `/v3/projects/853d712a78964af0ac45dcf7036c2681/users/6c15d52f83a24ab3ad2f821a52f7ccea/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.590711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.591350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.591632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.591904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.592578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.594645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.605280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=853d712a78964af0ac45dcf7036c2681, user_id=6c15d52f83a24ab3ad2f821a52f7ccea, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.607935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.726218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1049/2106] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:35.726218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.752371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:35.756902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1058/2107] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:35 2022] PUT /identity/v3/projects/853d712a78964af0ac45dcf7036c2681/users/6c15d52f83a24ab3ad2f821a52f7ccea/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:35.758138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:35.800120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedff4e9-0291-4797-a47e-d5afdc817975 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.833179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e096e-8ada-4e20-af72-631949f3dcc2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:35.838247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.838932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.839342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.839788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.840482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.842410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:35.862297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9848006-6dec-4466-b387-40cebf37dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:35.871168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848006-6dec-4466-b387-40cebf37dc23 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:35.871168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848006-6dec-4466-b387-40cebf37dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:35.871168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848006-6dec-4466-b387-40cebf37dc23 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:35.871168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9848006-6dec-4466-b387-40cebf37dc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:35.871168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9848006-6dec-4466-b387-40cebf37dc23 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.031759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1050/2108] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:36.031759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.031759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.031759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.035321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1059/2109] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:35 2022] POST /identity/v3/users => generated 290 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:36.036075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.039267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.053782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29c871-22a2-4dec-a268-901901185528 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.055087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:36.056371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.058660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.059300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.059585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.059939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] PATH_INFO: `/v3/projects/7045cbcdfcda411ba59d9e930410d6e0` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.061033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7045cbcdfcda411ba59d9e930410d6e0)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.065231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1051/2110] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:36.065380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.069149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.079667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29159c9-6a58-496c-af5f-cc67c395344a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.085509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.085509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.085966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.085966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.086513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.088413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.102108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1052/2111] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:36.102108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.111781 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fdacf-7a50-4ffc-a3fb-13a5a7c3e1cc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.116739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.117340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.117627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.118014 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.118683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.120277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.131234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1053/2112] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:36.132079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.133752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.186473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7045cbcdfcda411ba59d9e930410d6e0', 'request_id': 'req-8da33abb-1858-4337-a207-a66116c0be6c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:36.197503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca8cd7-16f4-45de-b3fe-5f2d43226ee3 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.247303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.247856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.248157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.248469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.283936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7045cbcdfcda411ba59d9e930410d6e0', 'request_id': 'req-8da33abb-1858-4337-a207-a66116c0be6c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:36.293691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7045cbcdfcda411ba59d9e930410d6e0', 'request_id': 'req-8da33abb-1858-4337-a207-a66116c0be6c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:36.315389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] The token cache is being invalidate because project 7045cbcdfcda411ba59d9e930410d6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:36.316024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:36.316916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:36.318729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1060/2113] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:36 2022] DELETE /identity/v3/projects/7045cbcdfcda411ba59d9e930410d6e0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:36.319848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.321908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.322594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.323573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.334676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1054/2114] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:36.335427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.338924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33abb-1858-4337-a207-a66116c0be6c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.344741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11f3d-e407-48f5-8aa2-fdc58f6133fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.345088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528441d-b7ff-4de4-a4bd-826ff337f50b service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.346271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11f3d-e407-48f5-8aa2-fdc58f6133fa admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.346861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11f3d-e407-48f5-8aa2-fdc58f6133fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.347383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11f3d-e407-48f5-8aa2-fdc58f6133fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.348180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca11f3d-e407-48f5-8aa2-fdc58f6133fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.350118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.350414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca11f3d-e407-48f5-8aa2-fdc58f6133fa admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.351439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.351722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.352041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.352770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.355690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.369973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1061/2115] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:36.369973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.369973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.369973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.370441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1055/2116] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:36.370441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.381231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c7617-087c-4663-b064-3b04a43bfae0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.386235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38a71c-2a1b-421e-b693-8e907ad12101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.386999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a71c-2a1b-421e-b693-8e907ad12101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.388400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a71c-2a1b-421e-b693-8e907ad12101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.388400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a71c-2a1b-421e-b693-8e907ad12101 admin admin] PATH_INFO: `/v3/projects/fbb2e61ecff9447e829247c05c9ec20e/users/8cf57a00ae81442e80688b5823fc6dd1/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.390979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:36.392279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.402064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1062/2117] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:36.402791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.404460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.405203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38a71c-2a1b-421e-b693-8e907ad12101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb2e61ecff9447e829247c05c9ec20e, user_id=8cf57a00ae81442e80688b5823fc6dd1, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.409057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38a71c-2a1b-421e-b693-8e907ad12101 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.417961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302d991-e568-411a-a1ce-cc3579e26f80 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.422809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1f9f4-f524-4013-ab46-5bcee536ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.424389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1f9f4-f524-4013-ab46-5bcee536ef59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.424741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1f9f4-f524-4013-ab46-5bcee536ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.424964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1f9f4-f524-4013-ab46-5bcee536ef59 admin admin] PATH_INFO: `/v3/projects/faf131654f5b49c4a4f987bdd6fef6e0/users/fa7ebe593403467f87b9e429203bb97b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.441291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1f9f4-f524-4013-ab46-5bcee536ef59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf131654f5b49c4a4f987bdd6fef6e0, user_id=fa7ebe593403467f87b9e429203bb97b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.444006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1f9f4-f524-4013-ab46-5bcee536ef59 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.487384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1056/2118] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:36 2022] PUT /identity/v3/projects/fbb2e61ecff9447e829247c05c9ec20e/users/8cf57a00ae81442e80688b5823fc6dd1/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:36.489029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff140cc-19d1-4961-b593-91869b5907c8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.489313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff140cc-19d1-4961-b593-91869b5907c8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.489589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff140cc-19d1-4961-b593-91869b5907c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.517110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1063/2119] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:36 2022] PUT /identity/v3/projects/faf131654f5b49c4a4f987bdd6fef6e0/users/fa7ebe593403467f87b9e429203bb97b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:36.519722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.519828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-5ff140cc-19d1-4961-b593-91869b5907c8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:36.520506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.520937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.521038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5ff140cc-19d1-4961-b593-91869b5907c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.555606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] MFA Rules not processed for user `8cf57a00ae81442e80688b5823fc6dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:36.557143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.605412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1057/2120] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:36.605412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.605412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.605412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.638670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] MFA Rules not processed for user `fa7ebe593403467f87b9e429203bb97b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:36.640046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:36.645135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1064/2121] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:36.645135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.660200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488c931-c245-4f32-89ab-fce907b4a082 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.674867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.676863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.678168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.678586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.679513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:36.679922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.681949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.703782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1065/2122] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:36.704694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.719457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a9606-1281-4872-b007-fcc23e2a0c21 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.727444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.728574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.728852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.729125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.729966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.736886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.738304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1058/2123] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:36.739145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.749765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178c7d-b681-4fba-aaea-0c83ff52e618 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.755541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.756286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.756595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.756991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.757671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.759576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.825861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1066/2124] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:36.825861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.885142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1059/2125] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:36.886111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:36.907754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b5bff-3d2b-4d7a-ab77-76551c029e69 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.955889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078c878-fd79-4332-930f-98726d24fae6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:36.960887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.961494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.961816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.962068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.962694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.964645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:36.966317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:36.966930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:36.967212 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:36.967485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:36.968118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:36.970038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.076007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1060/2126] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:37.079275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.087479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6edef0-8035-4001-84e9-1c58f2cd55ff admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.092408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.093043 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.093308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.093584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.094218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.097295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.114026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1067/2127] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:36 2022] POST /identity/v3/projects => generated 379 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:37.114026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.181282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063970b5-34ae-42d4-ad9e-08f8479559a6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.186035 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.186638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.186917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.187186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.187943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.190264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.200180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1068/2128] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:37.201138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.211373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408c387-b4d8-45e3-8b78-0253d70e70e6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.216172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.216904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.217169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.217443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.218087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.219978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.222508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1061/2129] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:37 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:37.223588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.245207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d55af-94d4-4dd9-95bd-2f05c35985ef admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.251099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.252219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.252710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.252943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.256700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.256700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.270368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1062/2130] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:37.270368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.280015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481aea0-be46-4253-ab90-8b5b41c43150 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.286119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.286969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.287533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.287669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.288387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.290156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.301270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1063/2131] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:37.301974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.313121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef379f7-aa12-445b-8eae-a0c61dc0567c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.320398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.325276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.325276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.325276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] PATH_INFO: `/v3/projects/36e2077819db4949a4c99bdc084d2c73/users/e3c64f1d95e24d55bc7eaf2be4c22bce/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.340763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e2077819db4949a4c99bdc084d2c73, user_id=e3c64f1d95e24d55bc7eaf2be4c22bce, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.344173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.352753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:37.359944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1069/2132] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:37 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:37.361025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.374337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2269c-685a-4ccc-932f-d697698171ff admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.380902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.381569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.381873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.382132 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.382797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.384458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.400452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1070/2133] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:37.400452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.418294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09d27-9477-4c5d-93b1-4136dde8a86f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.423023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.423653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.424559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.424887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.425558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.427211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.437661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1071/2134] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:37.438323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.444830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1064/2135] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:37 2022] PUT /identity/v3/projects/36e2077819db4949a4c99bdc084d2c73/users/e3c64f1d95e24d55bc7eaf2be4c22bce/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:37.445826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.449557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421dcd-7b63-47e8-bc9b-ef0221c6ad4c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.529134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ffa04-44f5-41b4-a54e-91d062acae9d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.531829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.532722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.533092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.533549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] PATH_INFO: `/v3/projects/3ca8eec30c0b4a659620b6f45bb94db9/users/14bd04e951254ec5bc29999bceeea214/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.534613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525db745-bcb5-4383-a922-f3b29442bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.535282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db745-bcb5-4383-a922-f3b29442bdea admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.535565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db745-bcb5-4383-a922-f3b29442bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.535843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db745-bcb5-4383-a922-f3b29442bdea admin admin] PATH_INFO: `/v3/projects/8234b28ae773496b93d1debae3cc002d/users/35ae4823094343888f9067841d6c32fd/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.552262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525db745-bcb5-4383-a922-f3b29442bdea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8234b28ae773496b93d1debae3cc002d, user_id=35ae4823094343888f9067841d6c32fd, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.553240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca8eec30c0b4a659620b6f45bb94db9, user_id=14bd04e951254ec5bc29999bceeea214, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.554888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525db745-bcb5-4383-a922-f3b29442bdea admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.556395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.647613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1065/2136] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:37 2022] PUT /identity/v3/projects/8234b28ae773496b93d1debae3cc002d/users/35ae4823094343888f9067841d6c32fd/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:37.651491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1072/2137] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:37 2022] PUT /identity/v3/projects/3ca8eec30c0b4a659620b6f45bb94db9/users/14bd04e951254ec5bc29999bceeea214/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:37.651628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d50b-c255-4b94-96f3-5221b1ae3ffd None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.651628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d50b-c255-4b94-96f3-5221b1ae3ffd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.651628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d50b-c255-4b94-96f3-5221b1ae3ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.651882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.678399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bfc2d50b-c255-4b94-96f3-5221b1ae3ffd None None] MFA Rules not processed for user `e3c64f1d95e24d55bc7eaf2be4c22bce`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:37.679843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bfc2d50b-c255-4b94-96f3-5221b1ae3ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:37.716759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05cbca-a27e-4662-9d24-31acfc14c12e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.750271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1066/2138] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:37.752186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.752653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.752824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.772353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:37.773579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:37.801032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.801738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.802022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.802352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.802953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.804570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.818709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1073/2139] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:37.824345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.836000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334886d-0f5c-427a-aa56-4b6cf9643413 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.841126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9e5c9-c7e0-4900-83cd-4a92c88c915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.841736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9e5c9-c7e0-4900-83cd-4a92c88c915f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.842012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9e5c9-c7e0-4900-83cd-4a92c88c915f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.842499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9e5c9-c7e0-4900-83cd-4a92c88c915f admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.843318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9e5c9-c7e0-4900-83cd-4a92c88c915f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.844951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9e5c9-c7e0-4900-83cd-4a92c88c915f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.856016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1067/2140] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:37.857216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.868823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76791f6e-8239-423f-9210-332ba6ece020 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.875706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:37.876615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.889774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.889774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] PATH_INFO: `/v3/projects/8234b28ae773496b93d1debae3cc002d/users/35ae4823094343888f9067841d6c32fd/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.889774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8234b28ae773496b93d1debae3cc002d, user_id=35ae4823094343888f9067841d6c32fd, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:37.889774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:37.893916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1068/2141] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:37 2022] PUT /identity/v3/projects/8234b28ae773496b93d1debae3cc002d/users/35ae4823094343888f9067841d6c32fd/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:37.894635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:37.897596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:37.956453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1074/2142] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:37 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:37.958850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737775-48dc-45b8-ac99-c24be52602d3 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:37.959239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737775-48dc-45b8-ac99-c24be52602d3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:37.959635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737775-48dc-45b8-ac99-c24be52602d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:37.959751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29eaf5-5a3a-4fca-86be-a4f98c8593ae admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:37.966654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:37.990903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2f737775-48dc-45b8-ac99-c24be52602d3 None None] MFA Rules not processed for user `6c15d52f83a24ab3ad2f821a52f7ccea`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:37.992235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2f737775-48dc-45b8-ac99-c24be52602d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:38.018259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.019045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.019326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.019599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.060368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1075/2143] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:38.061551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.061832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.062283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.081224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.083074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.091027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] MFA Rules not processed for user `35ae4823094343888f9067841d6c32fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:38.097272 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:38.097406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1069/2144] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:38.097406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.104399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20668e4e-e1dc-40fa-933e-789e76065ac3 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.109429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.110098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.110505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.110996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.111742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.115102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.163555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1076/2145] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:38.164810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.166159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:38.180943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc3413-1567-4b29-9c78-45c906afa220 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.185566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.186155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.186434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.186707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.187339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.189849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.207500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1070/2146] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:38 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:38.211921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.213628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:38.275214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dbd81-e615-4c00-956c-47fd55cc551c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.279692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.280511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.281373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.281751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.291320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1077/2147] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:38 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:38.292050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.292238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.293753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.304142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1071/2148] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:38.305041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.315457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd548-bc7d-40f0-8f2f-98c155fad84e service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.364800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd571-21d6-4eb7-b7b3-4c0ed0421a0f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.369587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.370279 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.370629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.370978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.371705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.373417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.374898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.375516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.375798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.376072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.377007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.378954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.384717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1078/2149] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:38.396964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.406790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f289e4-800c-4c0c-8aee-c89ef3e83d8b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.422987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.422987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.422987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.422987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] PATH_INFO: `/v3/projects/e4a54187fe114b489867b336e66ae646/users/4e6bada402a3433ab952142a066edfcf/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.446103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a54187fe114b489867b336e66ae646, user_id=4e6bada402a3433ab952142a066edfcf, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.450325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.490062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1072/2150] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:38 2022] POST /identity/v3/users => generated 309 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:38.498207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.508062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562a755-5a60-405f-ac77-2c61ea218cb0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.513024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.513630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.513910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.514184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.514918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.517736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.526341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1079/2151] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:38 2022] PUT /identity/v3/projects/e4a54187fe114b489867b336e66ae646/users/4e6bada402a3433ab952142a066edfcf/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:38.528340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1073/2152] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:05:38.533754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.534227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.544482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2851-a667-402e-b522-78e227f50d5f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.599454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53690-1a35-43fb-ba4d-da169c1c374f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.605426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.605840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.606099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.606383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.606656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.607311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.608944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.612990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.613343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.613723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] PATH_INFO: `/v3/projects/3d0dd39ae1fd4614ae990a450128903f/users/9253af911c3e47e9b0ad1a9d19f79cc5/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.623147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1080/2153] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:38.623147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.623437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:38.635638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0dd39ae1fd4614ae990a450128903f, user_id=9253af911c3e47e9b0ad1a9d19f79cc5, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.639157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.686417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b349b-3b78-4a0c-8c60-6f7da5d832b4 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.691649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.692426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.697250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.697250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.737929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1074/2154] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:38 2022] PUT /identity/v3/projects/3d0dd39ae1fd4614ae990a450128903f/users/9253af911c3e47e9b0ad1a9d19f79cc5/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:38.739150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.752786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.754329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.763873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1081/2155] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:38.765278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.776353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d3782-accc-4bbd-9fd7-bc55394b812f service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.811989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306f161-05be-41bf-8bf0-da34d824144d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.818422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a3070-0e82-49d0-b578-0db975fc2fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.819097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3070-0e82-49d0-b578-0db975fc2fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.819434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3070-0e82-49d0-b578-0db975fc2fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.819790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3070-0e82-49d0-b578-0db975fc2fa0 admin admin] PATH_INFO: `/v3/projects/e4a54187fe114b489867b336e66ae646/users/4e6bada402a3433ab952142a066edfcf/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.822423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a3070-0e82-49d0-b578-0db975fc2fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a54187fe114b489867b336e66ae646, user_id=4e6bada402a3433ab952142a066edfcf, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.825569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a3070-0e82-49d0-b578-0db975fc2fa0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.837774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1075/2156] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:38 2022] PUT /identity/v3/projects/e4a54187fe114b489867b336e66ae646/users/4e6bada402a3433ab952142a066edfcf/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:38.848203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.850771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.850771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.850936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.851836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.852341 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.852855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.853519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.862613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.868388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1082/2157] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:38.872014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:38.882194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] MFA Rules not processed for user `4e6bada402a3433ab952142a066edfcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:38.883570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751540d-abe3-4d3a-b0a2-32ebcba2d58b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:38.884643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:38.951280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf0f29-53c7-4f6b-9b05-7289bd178c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:38.952002 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0f29-53c7-4f6b-9b05-7289bd178c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:38.952288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0f29-53c7-4f6b-9b05-7289bd178c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:38.952833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0f29-53c7-4f6b-9b05-7289bd178c73 admin admin] PATH_INFO: `/v3/projects/3d0dd39ae1fd4614ae990a450128903f/users/9253af911c3e47e9b0ad1a9d19f79cc5/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:38.955385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf0f29-53c7-4f6b-9b05-7289bd178c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0dd39ae1fd4614ae990a450128903f, user_id=9253af911c3e47e9b0ad1a9d19f79cc5, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:38.958184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf0f29-53c7-4f6b-9b05-7289bd178c73 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:38.964675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1076/2158] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:38.969173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1083/2159] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:38 2022] PUT /identity/v3/projects/3d0dd39ae1fd4614ae990a450128903f/users/9253af911c3e47e9b0ad1a9d19f79cc5/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:38.977015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:39.035803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc9487-5057-44ac-921f-963444b3d068 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:39.079049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:39.084860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:39.085490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:39.085801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:39.086084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:39.147122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:39.148655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:39.158174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1077/2160] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:39.415549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:40.775793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:41.056849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:41.057442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:41.057890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:41.114371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] MFA Rules not processed for user `9253af911c3e47e9b0ad1a9d19f79cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:41.115695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:41.248892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1084/2161] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:41.495987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:42.424779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:42.439506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25761-b5af-40d9-ac88-cabcff803a05 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:42.443762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:42.444613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:42.444903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:42.445182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:42.508432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:42.509945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:42.519981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1078/2162] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:42.522477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:43.375058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:43.468758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66bcd-5d0b-4494-bb5a-d79ad71cdc4c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:43.479325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:43.480085 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:43.480477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:43.480822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] PATH_INFO: `/v3/users/6c15d52f83a24ab3ad2f821a52f7ccea` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:43.482066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c15d52f83a24ab3ad2f821a52f7ccea)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:43.484680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:43.785196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c15d52f83a24ab3ad2f821a52f7ccea'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:43.800053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c15d52f83a24ab3ad2f821a52f7ccea'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:43.904805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c15d52f83a24ab3ad2f821a52f7ccea'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:43.904805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c15d52f83a24ab3ad2f821a52f7ccea'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:43.929083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c15d52f83a24ab3ad2f821a52f7ccea'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:43.942214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1085/2163] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:43 2022] DELETE /identity/v3/users/6c15d52f83a24ab3ad2f821a52f7ccea => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:44.260120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:44.275457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405a0f4-b534-4b06-98b4-9d0a9056a5d0 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:44.344614 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:44.353160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:44.353817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:44.354102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:44.354383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] PATH_INFO: `/v3/projects/853d712a78964af0ac45dcf7036c2681` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:44.355628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=853d712a78964af0ac45dcf7036c2681)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:44.358073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:44.530461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '853d712a78964af0ac45dcf7036c2681', 'request_id': 'req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:44.644456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '853d712a78964af0ac45dcf7036c2681', 'request_id': 'req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:44.654075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '853d712a78964af0ac45dcf7036c2681', 'request_id': 'req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:44.666790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] The token cache is being invalidate because project 853d712a78964af0ac45dcf7036c2681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:44.667371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:44.668382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:44.670633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1079/2164] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:44 2022] DELETE /identity/v3/projects/853d712a78964af0ac45dcf7036c2681 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:44.675152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:44.676803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:44.691592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf925-177c-42a5-a18b-552184e0bb8c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:44.837841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:44.838507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:44.838768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:44.839043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] PATH_INFO: `/v3/users/9253af911c3e47e9b0ad1a9d19f79cc5` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:44.840468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9253af911c3e47e9b0ad1a9d19f79cc5)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:44.842456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:44.960302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:44.960302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.064995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486fe0f-d0a1-483a-895e-8f33bcb91d2b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.071318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.071961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.072456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.072735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.075153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.097124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9253af911c3e47e9b0ad1a9d19f79cc5'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.104747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9253af911c3e47e9b0ad1a9d19f79cc5'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.148554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.150117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e849a-909b-4597-9e59-65e18371fd73 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.163107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1080/2165] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:45.164366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.164864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.165199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.197143 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] MFA Rules not processed for user `14bd04e951254ec5bc29999bceeea214`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:45.198567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.200565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9253af911c3e47e9b0ad1a9d19f79cc5'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.201397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9253af911c3e47e9b0ad1a9d19f79cc5'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.220862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9253af911c3e47e9b0ad1a9d19f79cc5'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.228190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1086/2166] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:44 2022] DELETE /identity/v3/users/9253af911c3e47e9b0ad1a9d19f79cc5 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:45.286272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1081/2167] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:45.287229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.289187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.377148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251661c4-96ba-443d-9918-c249b2338c6e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.390943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.400621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.400621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.400621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.400621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.400621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.415313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:45.520198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1087/2168] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:45 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:45.520825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.522218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.524214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.545071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48d72-d94f-423d-8000-cafadccfaff0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.603945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0e1b3-a100-4994-a57d-a31f7e1e1974 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.608855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.609665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.610071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.610359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.611653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.613788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.613949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.614769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.615184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.615523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] PATH_INFO: `/v3/projects/3d0dd39ae1fd4614ae990a450128903f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.616909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0dd39ae1fd4614ae990a450128903f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.618950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.743553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1088/2169] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:45 2022] POST /identity/v3/users => generated 295 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:45.744616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.753498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0dd39ae1fd4614ae990a450128903f', 'request_id': 'req-cac28e84-476b-4de9-8083-e28474af0500'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.755131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b4152-02b4-42f1-9228-4704cc141bfa admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.823077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.823077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.823077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.823077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.823481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.823658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.835737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1089/2170] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:45.839444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0dd39ae1fd4614ae990a450128903f', 'request_id': 'req-cac28e84-476b-4de9-8083-e28474af0500'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.839988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.846679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0dd39ae1fd4614ae990a450128903f', 'request_id': 'req-cac28e84-476b-4de9-8083-e28474af0500'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.856903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05f9b9-f6b7-4b0a-9244-a303261f0038 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.858537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] The token cache is being invalidate because project 3d0dd39ae1fd4614ae990a450128903f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:45.859011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.859670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cac28e84-476b-4de9-8083-e28474af0500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:45.861687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1082/2171] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:45 2022] DELETE /identity/v3/projects/3d0dd39ae1fd4614ae990a450128903f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:45.862826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.863549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.863905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.864235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] PATH_INFO: `/v3/projects/d43808494dc44e9fbcc5b75e60c54b42/users/8cbcb286b19741a1a41a5b86b697a645/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.866586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665cbd4-cdb5-4d7c-9f47-7c62b9c7f076 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.867065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665cbd4-cdb5-4d7c-9f47-7c62b9c7f076 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.867193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665cbd4-cdb5-4d7c-9f47-7c62b9c7f076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.882466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d43808494dc44e9fbcc5b75e60c54b42, user_id=8cbcb286b19741a1a41a5b86b697a645, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.885638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.889244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-6665cbd4-cdb5-4d7c-9f47-7c62b9c7f076 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:45.890565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6665cbd4-cdb5-4d7c-9f47-7c62b9c7f076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.899728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1083/2172] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:45.906956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.907250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.907518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.926541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:45.928461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:45.929729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.937750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1084/2173] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:45.942015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.950848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50766-c47c-466a-b0c6-d042332b2a8f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:45.956062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:45.956689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:45.956964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:45.957415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:45.958076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:45.958412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:45.960341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:45.963281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1090/2174] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:45 2022] PUT /identity/v3/projects/d43808494dc44e9fbcc5b75e60c54b42/users/8cbcb286b19741a1a41a5b86b697a645/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:45.976462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:45.979749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:45.982601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1085/2175] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:05:45.987234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.005476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362f936-1930-4dc0-8cae-5ea888ab8691 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.069867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075d599-e709-4974-a0f9-b962eb74365e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.075297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.075915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.076198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.076434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.076544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.076965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.077187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.077288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.077542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.078189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.079184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.080639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.091282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1091/2176] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:46.091973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.093673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:46.108394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270d32b-4035-48ae-b434-8f59de3f683c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.114941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.115576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.115858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.116131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] PATH_INFO: `/v3/users/f1724a8c8b764ab2b828cf32b9299c1c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.117304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1724a8c8b764ab2b828cf32b9299c1c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.119230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.167730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1086/2177] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:45 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:05:46.168832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.231267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038becb-67a2-403e-9100-de5078c4b51f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.236379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.240683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.240683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.240683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] PATH_INFO: `/v3/projects/d43808494dc44e9fbcc5b75e60c54b42/users/8cbcb286b19741a1a41a5b86b697a645/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.240683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d43808494dc44e9fbcc5b75e60c54b42, user_id=8cbcb286b19741a1a41a5b86b697a645, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.242508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.254501 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1087/2178] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:46 2022] PUT /identity/v3/projects/d43808494dc44e9fbcc5b75e60c54b42/users/8cbcb286b19741a1a41a5b86b697a645/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:46.255332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.272761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1724a8c8b764ab2b828cf32b9299c1c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:46.281668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1724a8c8b764ab2b828cf32b9299c1c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:46.324823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fa73-c83f-4fa1-ad3c-d6c1ef936a20 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.330419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.331138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.331418 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.331687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.332375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.334323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.386525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1724a8c8b764ab2b828cf32b9299c1c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:46.387048 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1724a8c8b764ab2b828cf32b9299c1c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:46.418232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bf8ae250-78cb-436d-bc68-322d9c64a293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1724a8c8b764ab2b828cf32b9299c1c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:46.436537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1088/2179] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:46 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:46.438113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1092/2180] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:46 2022] DELETE /identity/v3/users/f1724a8c8b764ab2b828cf32b9299c1c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:46.443209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.445996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:46.447817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.449358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.449756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.496882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b570-6559-42af-accd-a3e7d2eba4a0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.520149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:46.521836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:46.560537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.561166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.561450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.561727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.562382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.565577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.575670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1089/2181] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:46.579915 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.590460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd50f8-faf1-440a-9319-6c90c95dc274 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.593923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1093/2182] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:46.599129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.599728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.600065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.600360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.601019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.602609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.617748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.618051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1090/2183] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:05:46.621590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:46.622433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:46.633409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6852f5-72d3-4c74-9fbd-a15c0811a7fd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.638623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.639336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.639618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.639890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] PATH_INFO: `/v3/projects/d0590c1a178a4675a0dc0089e3e92f05/users/cafe31db79ce4841810b91cc0b0e4a32/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.656142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0590c1a178a4675a0dc0089e3e92f05, user_id=cafe31db79ce4841810b91cc0b0e4a32, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.662035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.704196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcf920-5923-4d77-abb9-0d968c82efbd None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:46.709665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2dc7e-0e78-41bc-96a3-439a866056a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:46.710891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2dc7e-0e78-41bc-96a3-439a866056a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.711425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2dc7e-0e78-41bc-96a3-439a866056a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.712697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2dc7e-0e78-41bc-96a3-439a866056a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.725021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2dc7e-0e78-41bc-96a3-439a866056a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:46.727177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2dc7e-0e78-41bc-96a3-439a866056a5 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:46.741987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1094/2184] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:46.790460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1091/2185] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:46 2022] PUT /identity/v3/projects/d0590c1a178a4675a0dc0089e3e92f05/users/cafe31db79ce4841810b91cc0b0e4a32/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:46.791458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131eefc-f339-4613-b937-761e74be6079 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:46.791749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131eefc-f339-4613-b937-761e74be6079 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:46.792024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131eefc-f339-4613-b937-761e74be6079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:46.839636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-2131eefc-f339-4613-b937-761e74be6079 None None] MFA Rules not processed for user `cafe31db79ce4841810b91cc0b0e4a32`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:46.841175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-2131eefc-f339-4613-b937-761e74be6079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:46.931742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1095/2186] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:46.932982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.017390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226312d5-a8a5-4fc2-9f67-81b61e03b16f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.022485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.023162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.023446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.023726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.024419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.026299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.163892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1092/2187] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:46 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:47.168122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131eefc-f339-4613-b937-761e74be6079 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.180821 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131eefc-f339-4613-b937-761e74be6079 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.256171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.256899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.257183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.257460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.258125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.260071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.297158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.299466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:47.320002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807e6da-4ee9-4b27-a35a-ed074c804e4a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.328258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.330328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.331073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.331503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] PATH_INFO: `/v3/projects/68628da9dfef45c8964e0ea546d37270` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.333463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68628da9dfef45c8964e0ea546d37270)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.341361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.390902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1096/2188] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:47 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:05:47.395724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.408952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537bafa-30b4-4b26-847a-35334e578237 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.412177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:47.417414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.421945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.421945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.421945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.421945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.421945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.437158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1097/2189] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:47.438079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.451792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b205-170e-423d-9665-f00d09fef59f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.458715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.459511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.459850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.460209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] PATH_INFO: `/v3/projects/bcac56fee80c43f18bc696950ed9447f/users/f0d4a645903549e2b062667771999a9e/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.477053 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcac56fee80c43f18bc696950ed9447f, user_id=f0d4a645903549e2b062667771999a9e, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.479712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.490998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68628da9dfef45c8964e0ea546d37270', 'request_id': 'req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:47.602517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68628da9dfef45c8964e0ea546d37270', 'request_id': 'req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:47.602898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:47.614740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1098/2190] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:47 2022] PUT /identity/v3/projects/bcac56fee80c43f18bc696950ed9447f/users/f0d4a645903549e2b062667771999a9e/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:47.619724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.622272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68628da9dfef45c8964e0ea546d37270', 'request_id': 'req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:47.646453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] The token cache is being invalidate because project 68628da9dfef45c8964e0ea546d37270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:47.648513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:47.652237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9b51d9d8-c4b4-43ee-9924-5b8fc095a4a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:47.656556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1093/2191] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:47 2022] DELETE /identity/v3/projects/68628da9dfef45c8964e0ea546d37270 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:47.663141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ed45-073d-4969-b7dc-62d08b5c5178 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.663141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ed45-073d-4969-b7dc-62d08b5c5178 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.663141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ed45-073d-4969-b7dc-62d08b5c5178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.692204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d901ed45-073d-4969-b7dc-62d08b5c5178 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:47.693827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d901ed45-073d-4969-b7dc-62d08b5c5178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:47.750771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7dc0f-40ee-413a-a776-3ca2dfb86e6e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.777114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.777745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.778059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.778334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.778999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.780788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.796774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1099/2192] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:47.801758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.803601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:47.810919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1094/2193] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:47.812932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.813414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.815940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.820682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5ee01-f60e-4026-adff-5eb215fef707 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.827467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739d86-9f6c-4269-9c53-42958a78ac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.828150 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739d86-9f6c-4269-9c53-42958a78ac78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.828490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739d86-9f6c-4269-9c53-42958a78ac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.828766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739d86-9f6c-4269-9c53-42958a78ac78 admin admin] PATH_INFO: `/v3/projects/bcac56fee80c43f18bc696950ed9447f/users/f0d4a645903549e2b062667771999a9e/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.830828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739d86-9f6c-4269-9c53-42958a78ac78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcac56fee80c43f18bc696950ed9447f, user_id=f0d4a645903549e2b062667771999a9e, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.834610 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739d86-9f6c-4269-9c53-42958a78ac78 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:47.847673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:47.849755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:47.851501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1100/2194] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:47 2022] PUT /identity/v3/projects/bcac56fee80c43f18bc696950ed9447f/users/f0d4a645903549e2b062667771999a9e/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:47.872966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.876920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.877502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.916901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] MFA Rules not processed for user `f0d4a645903549e2b062667771999a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:47.920461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:47.936605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1095/2195] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:47.945416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:47.970587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b78a5-3181-4f93-9dca-ebb0336d3c31 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:47.986907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:47.987992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:47.987992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:47.988262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:47.988973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:47.989283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:47.990895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.014780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1096/2196] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:05:48.023331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1101/2197] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:48.025989 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:48.036653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:48.038171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:48.056398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e2e33-e7b0-4945-831f-e7be2c766c36 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:48.065304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:48.065934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.066215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.066633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.067288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:48.069242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.127566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c093d-0e23-4680-aa3d-4171522c75f2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:48.132794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:48.133561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.133952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.134329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.145946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:48.147609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.159636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1102/2198] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:48.227096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1097/2199] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:48 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:48.228231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:48.261849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d04e3-4b89-472e-9540-67b9c208aa83 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:48.347158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:48.348087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.349038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.350555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.351369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:48.353836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.493576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1103/2200] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:48 2022] POST /identity/v3/users => generated 288 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:48.494831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:48.506450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb074b9-cd13-4531-8bf2-4d46de5689df admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:48.513908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:48.514560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.514840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.515112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.515773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:48.518047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.530166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1098/2201] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:48.534399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:48.545782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c8af8-93da-4575-b5e9-e9eed0e56c0c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:48.551358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89aad2-c6d4-4422-b917-36684a85bdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:48.552193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89aad2-c6d4-4422-b917-36684a85bdcf admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.552769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89aad2-c6d4-4422-b917-36684a85bdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.553125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89aad2-c6d4-4422-b917-36684a85bdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.554024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89aad2-c6d4-4422-b917-36684a85bdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:48.555895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89aad2-c6d4-4422-b917-36684a85bdcf admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.569318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1104/2202] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:05:48.573637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:48.587358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2b157-a64d-4940-b79a-3f2ec751f06f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:48.600983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:48.601670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.601948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.602220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] PATH_INFO: `/v3/projects/8c5fa871ae8a46cbac26cc1b8646f4ba/users/06c3e87487c34845bfb24ceccf0aff7c/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.648263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5fa871ae8a46cbac26cc1b8646f4ba, user_id=06c3e87487c34845bfb24ceccf0aff7c, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:48.655300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:48.784765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:48.793631 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1099/2203] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:48 2022] PUT /identity/v3/projects/8c5fa871ae8a46cbac26cc1b8646f4ba/users/06c3e87487c34845bfb24ceccf0aff7c/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:48.804424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:48.804715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:48.804990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:48.844107 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] MFA Rules not processed for user `06c3e87487c34845bfb24ceccf0aff7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:48.846096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:48.943457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1105/2204] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:48.953835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:49.047357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373a113-bcd9-49e5-bebe-0e39d50b1343 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:49.052646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:49.053916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:49.053916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:49.054137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:49.064983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:49.066552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:49.081181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1100/2205] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:49.474696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:49.557886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd7f06-900a-4684-af8b-b6d40aebe9c0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:49.562776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d4a645903549e2b062667771999a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcac56fee80c43f18bc696950ed9447f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:49.565173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:49.565173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:49.565173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:49.565692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:49.566003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:49.567648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:49.583492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1106/2206] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:49.584682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:49.596596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702718f2-40a0-49ab-9c7d-c9295d567816 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:49.601245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d4a645903549e2b062667771999a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcac56fee80c43f18bc696950ed9447f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:49.601924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:49.602217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:49.602497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:49.603150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:49.605400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:49.783225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1101/2207] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:49 2022] POST /identity/v3/projects => generated 354 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:49.793597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:49.808027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b80f1-88f8-4402-8249-1c4f6854827d tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:49.873848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d4a645903549e2b062667771999a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcac56fee80c43f18bc696950ed9447f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:49.881364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:49.881364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:49.881364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:49.881364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:49.881364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:49.881938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:49.891979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1107/2208] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:50.383829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:50.461208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:50.481399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276729c7-45b2-4e35-ad82-e30c641cf1fe tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:50.489641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0d4a645903549e2b062667771999a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcac56fee80c43f18bc696950ed9447f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:50.491038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:50.491392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:50.491700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] PATH_INFO: `/v3/projects/d148f5dff0e34620831796d1d84a571c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:50.493560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d148f5dff0e34620831796d1d84a571c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:50.495825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:50.704130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd148f5dff0e34620831796d1d84a571c', 'request_id': 'req-f4724f85-5db2-4b25-bfe9-9d933ff1655c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:50.809870 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd148f5dff0e34620831796d1d84a571c', 'request_id': 'req-f4724f85-5db2-4b25-bfe9-9d933ff1655c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:50.823446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd148f5dff0e34620831796d1d84a571c', 'request_id': 'req-f4724f85-5db2-4b25-bfe9-9d933ff1655c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:50.846090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] The token cache is being invalidate because project d148f5dff0e34620831796d1d84a571c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:50.847364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:50.849834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f4724f85-5db2-4b25-bfe9-9d933ff1655c tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:50.856661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1102/2209] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:50 2022] DELETE /identity/v3/projects/d148f5dff0e34620831796d1d84a571c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:51.090542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:51.848552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:51.850979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:51.944251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb20af-beb1-46ed-ab04-7bffc4183f74 tempest-VolumeQuotaClassesTest-247935329 tempest-VolumeQuotaClassesTest-247935329-project-admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:52.001089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:52.002463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:52.002745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:52.003145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] PATH_INFO: `/v3/users/cafe31db79ce4841810b91cc0b0e4a32` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:52.004303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafe31db79ce4841810b91cc0b0e4a32)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:52.006227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:52.203210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafe31db79ce4841810b91cc0b0e4a32'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:52.211094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafe31db79ce4841810b91cc0b0e4a32'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:52.297888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafe31db79ce4841810b91cc0b0e4a32'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:52.298556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafe31db79ce4841810b91cc0b0e4a32'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:52.314400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafe31db79ce4841810b91cc0b0e4a32'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:52.324790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1108/2210] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:51 2022] DELETE /identity/v3/users/cafe31db79ce4841810b91cc0b0e4a32 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:52.326572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:52.327040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:52.327172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:52.348175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:52.349969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:52.383930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:52.429271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1103/2211] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:52.437271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:52.439633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:52.522926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be8574-5fff-4de2-81c7-0f9a44eba5e6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:52.529050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639717e-df3c-43aa-b903-38f1b0b8d00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:52.529896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639717e-df3c-43aa-b903-38f1b0b8d00f service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:52.530227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639717e-df3c-43aa-b903-38f1b0b8d00f service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:52.533361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639717e-df3c-43aa-b903-38f1b0b8d00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:52.548885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639717e-df3c-43aa-b903-38f1b0b8d00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:52.551590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639717e-df3c-43aa-b903-38f1b0b8d00f service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:52.566539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1109/2212] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:53.139579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:53.289384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:53.290904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:53.295809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:53.296251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:53.296434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:53.303746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a08c-9d9c-48cc-bc02-98e6f0401826 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:53.312991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:53.313878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:53.314163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:53.314446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] PATH_INFO: `/v3/projects/d0590c1a178a4675a0dc0089e3e92f05` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:53.319584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0590c1a178a4675a0dc0089e3e92f05)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:53.321670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:53.370975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] MFA Rules not processed for user `8cbcb286b19741a1a41a5b86b697a645`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:53.370975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:53.459069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1110/2213] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:53.475170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:53.476614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:53.528158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0590c1a178a4675a0dc0089e3e92f05', 'request_id': 'req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:53.554496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a64454-5a07-4b7a-b1d3-a013480d5570 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:53.617206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0590c1a178a4675a0dc0089e3e92f05', 'request_id': 'req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:53.622415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb89411-8be4-4227-8300-f3118285802f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:53.623042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb89411-8be4-4227-8300-f3118285802f service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:53.623325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb89411-8be4-4227-8300-f3118285802f service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:53.623599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb89411-8be4-4227-8300-f3118285802f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:53.624517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0590c1a178a4675a0dc0089e3e92f05', 'request_id': 'req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:53.625458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-beb89411-8be4-4227-8300-f3118285802f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:53.642332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] The token cache is being invalidate because project d0590c1a178a4675a0dc0089e3e92f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:53.642708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:53.643604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:53.645684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1104/2214] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:53 2022] DELETE /identity/v3/projects/d0590c1a178a4675a0dc0089e3e92f05 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:53.649348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:53.651789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:53.691697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb89411-8be4-4227-8300-f3118285802f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:53.693472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb89411-8be4-4227-8300-f3118285802f service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:53.705119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1111/2215] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:53.772897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6ea12-1434-4fcd-a4ba-138e33ee5e40 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:53.786269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:53.786925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:53.787206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:53.787513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] PATH_INFO: `/v3/users/f0d4a645903549e2b062667771999a9e` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:53.791595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d4a645903549e2b062667771999a9e)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:53.793681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:54.120518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d4a645903549e2b062667771999a9e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:54.128729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d4a645903549e2b062667771999a9e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:54.129997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d4a645903549e2b062667771999a9e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:54.130723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d4a645903549e2b062667771999a9e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:54.226330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d4a645903549e2b062667771999a9e'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:54.246446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1105/2216] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:53 2022] DELETE /identity/v3/users/f0d4a645903549e2b062667771999a9e => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:54.869988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb89411-8be4-4227-8300-f3118285802f service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:54.887027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb89411-8be4-4227-8300-f3118285802f service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:54.954290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:55.008969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:55.009611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.009886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.010156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] PATH_INFO: `/v3/projects/bcac56fee80c43f18bc696950ed9447f` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.011547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcac56fee80c43f18bc696950ed9447f)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:55.013682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:55.252936 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:55.254435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:55.269154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa453-385b-40e2-bc0a-6437a4e58c6f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:55.273879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:55.274524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.274782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.275060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] PATH_INFO: `/v3/users/8cf57a00ae81442e80688b5823fc6dd1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.276204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf57a00ae81442e80688b5823fc6dd1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:55.278141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:55.479163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcac56fee80c43f18bc696950ed9447f', 'request_id': 'req-a1d71a1a-17b7-4579-a848-4561008a090b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.547251 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcac56fee80c43f18bc696950ed9447f', 'request_id': 'req-a1d71a1a-17b7-4579-a848-4561008a090b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.556820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcac56fee80c43f18bc696950ed9447f', 'request_id': 'req-a1d71a1a-17b7-4579-a848-4561008a090b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.579881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] The token cache is being invalidate because project bcac56fee80c43f18bc696950ed9447f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:55.580248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.581137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.586442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1112/2217] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:54 2022] DELETE /identity/v3/projects/bcac56fee80c43f18bc696950ed9447f => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:55.587366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:55.588844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:55.640406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf57a00ae81442e80688b5823fc6dd1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.648100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf57a00ae81442e80688b5823fc6dd1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.649044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf57a00ae81442e80688b5823fc6dd1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.649750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf57a00ae81442e80688b5823fc6dd1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.664418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d71a1a-17b7-4579-a848-4561008a090b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:55.669400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:55.670057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.670317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.670602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] PATH_INFO: `/v3/users/fa7ebe593403467f87b9e429203bb97b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.671740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7ebe593403467f87b9e429203bb97b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:55.673715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:55.711813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f3f6f397-46aa-4329-b42e-d5d7e1f42294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf57a00ae81442e80688b5823fc6dd1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.734123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1106/2218] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:55 2022] DELETE /identity/v3/users/8cf57a00ae81442e80688b5823fc6dd1 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:55.735465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029db6-122a-40e7-8657-32777dd8ed80 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.735772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029db6-122a-40e7-8657-32777dd8ed80 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.736044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029db6-122a-40e7-8657-32777dd8ed80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.755701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ac029db6-122a-40e7-8657-32777dd8ed80 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:55.758628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ac029db6-122a-40e7-8657-32777dd8ed80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:55.832962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1107/2219] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:55.832962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8810f-28e5-4d21-816c-9a43174ef793 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.832962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8810f-28e5-4d21-816c-9a43174ef793 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.832962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8810f-28e5-4d21-816c-9a43174ef793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.841284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-8bf8810f-28e5-4d21-816c-9a43174ef793 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:55.842697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8bf8810f-28e5-4d21-816c-9a43174ef793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:55.850894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1108/2220] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:55.855103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.855395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.855670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.877472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:55.878736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:55.890641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1109/2221] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:55.890641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:55.890641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:55.917556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7ebe593403467f87b9e429203bb97b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.928661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7ebe593403467f87b9e429203bb97b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:55.937507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:55.969036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec40ab-3c94-4b0b-92e5-2862a5faac83 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:55.973344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:55.974023 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:55.974496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:55.974770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:55.984196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:55.991624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.007700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1110/2222] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:05:56.008887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.019218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeebb5-b4f9-45a6-86c2-128b8bb35f79 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.025135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7ebe593403467f87b9e429203bb97b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:56.026117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7ebe593403467f87b9e429203bb97b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:56.041217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.042069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.042353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.042627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.043299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:05:56.043591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.047011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.049775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-919931c4-740e-4c6b-95a3-47f58e2e735e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7ebe593403467f87b9e429203bb97b'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:56.059632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1113/2223] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:55 2022] DELETE /identity/v3/users/fa7ebe593403467f87b9e429203bb97b => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:56.065275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1111/2224] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:05:56.065524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.065524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.065524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.067327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.070852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:56.086882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:56.088184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875bf1d-918f-4fa5-9d54-939d12795bfd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.088639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:56.181871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.183051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.183540 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.183970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.184832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.187671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.206724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1114/2225] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:56.215196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.216784 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:56.304225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4606-2787-4f69-b5a4-8b50509eb942 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.307822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.308477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.308759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.309033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.318866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.320283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.330389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1115/2226] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:05:56.337602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1112/2227] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:56 2022] POST /identity/v3/projects => generated 385 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:56.349536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.371038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c565b47-1a8d-4604-adbb-0d8fec24c744 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.458420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.459037 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.459316 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.459586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.460285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.462394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.608746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1116/2228] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:56 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:56.611199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.622773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35f11-c74b-4335-8fd2-544978353e45 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.628429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.629473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.629803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.630139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.630948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.634920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.657283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1113/2229] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:56.670822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.675020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaeea3-9a84-45e8-8dfc-a287bbee8c90 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.680873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.681617 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.682000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.682593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.683596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.685534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.713643 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1117/2230] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:05:56.717749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.738572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03623535-2221-419c-8f83-3dc4b858c1b8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.758787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:56.765331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:56.793835 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d815e5-1453-4a8a-9b41-c4b9dd132ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.806679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d815e5-1453-4a8a-9b41-c4b9dd132ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.806679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d815e5-1453-4a8a-9b41-c4b9dd132ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.806679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d815e5-1453-4a8a-9b41-c4b9dd132ef7 admin admin] PATH_INFO: `/v3/projects/366b44bbf20340d7b1924ff1543a3638/users/7d4abb23a1d94b39b9b14422c9a2eeb7/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.821200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfb583-a888-4e38-b325-09491316f246 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:56.840641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:56.841562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:56.841824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:56.842165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] PATH_INFO: `/v3/projects/fbb2e61ecff9447e829247c05c9ec20e` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:56.844162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb2e61ecff9447e829247c05c9ec20e)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.848793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.857618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d815e5-1453-4a8a-9b41-c4b9dd132ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=366b44bbf20340d7b1924ff1543a3638, user_id=7d4abb23a1d94b39b9b14422c9a2eeb7, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:56.864188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d815e5-1453-4a8a-9b41-c4b9dd132ef7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:56.988968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1114/2231] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:56 2022] PUT /identity/v3/projects/366b44bbf20340d7b1924ff1543a3638/users/7d4abb23a1d94b39b9b14422c9a2eeb7/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:57.001895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:57.002662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:57.003313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:57.043935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb2e61ecff9447e829247c05c9ec20e', 'request_id': 'req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.051698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] MFA Rules not processed for user `7d4abb23a1d94b39b9b14422c9a2eeb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:57.053153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:57.121997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1115/2232] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:05:57.125306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:57.127253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:57.190878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001cf3c-c31f-4879-848d-2017d6fc49ef None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:57.195564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:57.196230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:57.196543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:57.196817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] PATH_INFO: `/v3/projects/faf131654f5b49c4a4f987bdd6fef6e0` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:57.197940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf131654f5b49c4a4f987bdd6fef6e0)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:57.199854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:57.207632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:57.211027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb2e61ecff9447e829247c05c9ec20e', 'request_id': 'req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.227832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb2e61ecff9447e829247c05c9ec20e', 'request_id': 'req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.257738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] The token cache is being invalidate because project fbb2e61ecff9447e829247c05c9ec20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:57.258828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.259663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.261234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1118/2233] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:56 2022] DELETE /identity/v3/projects/fbb2e61ecff9447e829247c05c9ec20e => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:05:57.264090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:57.264090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:57.277059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea6b70-011f-4de1-b914-18a5d5e8ffc2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:57.281771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:57.283867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:57.283867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:57.283867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:57.283867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:57.285916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:57.341497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf131654f5b49c4a4f987bdd6fef6e0', 'request_id': 'req-9741bfd8-eba6-40f0-833e-612303c76ce0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.418064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf131654f5b49c4a4f987bdd6fef6e0', 'request_id': 'req-9741bfd8-eba6-40f0-833e-612303c76ce0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.434470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf131654f5b49c4a4f987bdd6fef6e0', 'request_id': 'req-9741bfd8-eba6-40f0-833e-612303c76ce0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.453617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] The token cache is being invalidate because project faf131654f5b49c4a4f987bdd6fef6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:57.454958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.455960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1119/2234] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:57 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:57.456757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.457393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:57.459569 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1116/2235] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:57 2022] DELETE /identity/v3/projects/faf131654f5b49c4a4f987bdd6fef6e0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:57.460753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:57.463111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:57.464817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:57.618355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741bfd8-eba6-40f0-833e-612303c76ce0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:57.630457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:57.632054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:57.640538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:57.640538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:57.640538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:57.640538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:57.648047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72415c9-1a15-495e-8f29-0c03eadb7e99 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:57.674654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:57.676602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:57.676887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:57.677167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] PATH_INFO: `/v3/users/e3c64f1d95e24d55bc7eaf2be4c22bce` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:57.679723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c64f1d95e24d55bc7eaf2be4c22bce)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:57.684635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:57.739258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1117/2236] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:57 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:57.740798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:57.745588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:57.765210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03790403-4cd1-4764-9b3b-9622646c6d58 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:57.769821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:57.774662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:57.775464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:57.775927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:57.776377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] PATH_INFO: `/v3/users/35ae4823094343888f9067841d6c32fd` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:57.777716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ae4823094343888f9067841d6c32fd)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:57.781906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:57.989087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c64f1d95e24d55bc7eaf2be4c22bce'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:57.998644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c64f1d95e24d55bc7eaf2be4c22bce'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.039516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ae4823094343888f9067841d6c32fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.047721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ae4823094343888f9067841d6c32fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.048531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ae4823094343888f9067841d6c32fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.049248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ae4823094343888f9067841d6c32fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.132595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c64f1d95e24d55bc7eaf2be4c22bce'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.133292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c64f1d95e24d55bc7eaf2be4c22bce'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.148554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c64f1d95e24d55bc7eaf2be4c22bce'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.156226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1120/2237] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:57 2022] DELETE /identity/v3/users/e3c64f1d95e24d55bc7eaf2be4c22bce => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:58.160438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:58.166890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:05:58.176561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:58.211754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ae4823094343888f9067841d6c32fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:58.241744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1118/2238] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:57 2022] DELETE /identity/v3/users/35ae4823094343888f9067841d6c32fd => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:58.275431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75072e15-7235-44a2-a53c-ac8d54ee1217 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:58.340149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:58.341015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:58.341422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:58.341724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:58.342431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:58.344067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:58.354636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1121/2239] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:58.358682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:58.360106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:58.373442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29456757-189c-4f66-a1d7-0412037b1bcc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:58.382630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c804d59-b18b-45bd-9f42-08d8f045997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:58.382630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804d59-b18b-45bd-9f42-08d8f045997f admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:58.382630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804d59-b18b-45bd-9f42-08d8f045997f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:58.382630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c804d59-b18b-45bd-9f42-08d8f045997f admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:58.383020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c804d59-b18b-45bd-9f42-08d8f045997f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:58.383020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c804d59-b18b-45bd-9f42-08d8f045997f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:58.396152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1119/2240] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:58.397819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:58.408115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee91c8-d5fc-4936-b1ab-c329893889b2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:58.413264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:58.413936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:58.414214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:58.414489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] PATH_INFO: `/v3/projects/40d0a7e809a04755b53274e3d7ecbfc1/users/9e62be781db44b89955da9d06ceafaf6/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:58.431306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d0a7e809a04755b53274e3d7ecbfc1, user_id=9e62be781db44b89955da9d06ceafaf6, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:58.434063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:58.538378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1122/2241] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:58 2022] PUT /identity/v3/projects/40d0a7e809a04755b53274e3d7ecbfc1/users/9e62be781db44b89955da9d06ceafaf6/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:58.548618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:58.549078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:58.549431 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:58.632795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] MFA Rules not processed for user `9e62be781db44b89955da9d06ceafaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:58.637419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:58.719336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:58.719336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:58.850712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1120/2242] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:58.857840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:58.882073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac662bdc-e3e1-4ffe-b5b9-7542de04cdc2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:58.883128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1c06-de15-4123-bbae-b35ea57f3a25 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:58.891285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:58.892976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:58.893365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:58.893738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] PATH_INFO: `/v3/projects/36e2077819db4949a4c99bdc084d2c73` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:58.894395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:58.895005 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:58.895287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:58.895561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:58.895681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e2077819db4949a4c99bdc084d2c73)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:58.896208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:58.898354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:58.900095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.081589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1121/2243] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:05:58 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:59.081589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.081589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.097433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e2077819db4949a4c99bdc084d2c73', 'request_id': 'req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.150581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4bb70-e700-4dee-b305-63c795970418 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.220484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.221132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.221414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.221690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] PATH_INFO: `/v3/projects/8234b28ae773496b93d1debae3cc002d` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.222846 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8234b28ae773496b93d1debae3cc002d)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.224944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.234926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e2077819db4949a4c99bdc084d2c73', 'request_id': 'req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.243682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e2077819db4949a4c99bdc084d2c73', 'request_id': 'req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.256847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] The token cache is being invalidate because project 36e2077819db4949a4c99bdc084d2c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:59.256847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.257289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.259052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1123/2244] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:58 2022] DELETE /identity/v3/projects/36e2077819db4949a4c99bdc084d2c73 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:59.259905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.262327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.277381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab97b80-eb1e-4e00-832b-f03f3ca97a9a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.282519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.283193 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.283471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.283826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.284652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.286489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.361472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8234b28ae773496b93d1debae3cc002d', 'request_id': 'req-a49b6e2e-0063-489a-9dff-1c9a55e338c2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.386733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1124/2245] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:05:59 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:05:59.392888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.392888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.408091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1172b-1c5a-4e22-8c5e-52fb8165dd9c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.442551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8234b28ae773496b93d1debae3cc002d', 'request_id': 'req-a49b6e2e-0063-489a-9dff-1c9a55e338c2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.451571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8234b28ae773496b93d1debae3cc002d', 'request_id': 'req-a49b6e2e-0063-489a-9dff-1c9a55e338c2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.472735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] The token cache is being invalidate because project 8234b28ae773496b93d1debae3cc002d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:05:59.477332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.477332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.477332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1122/2246] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:05:59 2022] DELETE /identity/v3/projects/8234b28ae773496b93d1debae3cc002d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:59.477332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.478019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.481242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.481974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.482353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.482733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] PATH_INFO: `/v3/users/4e6bada402a3433ab952142a066edfcf` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.484479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6bada402a3433ab952142a066edfcf)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.487401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.496207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b6e2e-0063-489a-9dff-1c9a55e338c2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.502021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a4c73-611a-4dc2-8f91-7f7747d19488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.502653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a4c73-611a-4dc2-8f91-7f7747d19488 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.502950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a4c73-611a-4dc2-8f91-7f7747d19488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.503216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a4c73-611a-4dc2-8f91-7f7747d19488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.504087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a4c73-611a-4dc2-8f91-7f7747d19488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.508387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a4c73-611a-4dc2-8f91-7f7747d19488 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.525925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1123/2247] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:59.525925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.525925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.525925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.549476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:59.550820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.559735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1124/2248] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:59.560609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.576807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a979aa-2e9a-443d-8b70-d08c2a76f89e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.582304 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57f743-fdea-43b6-808c-250886e56780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.582966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57f743-fdea-43b6-808c-250886e56780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.583248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57f743-fdea-43b6-808c-250886e56780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.583644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57f743-fdea-43b6-808c-250886e56780 admin admin] PATH_INFO: `/v3/projects/0f036ea7c0e94bb291d680aee2dae16d/users/5dda3c6f56c14b2ea40e594664dbfd2c/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.600098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57f743-fdea-43b6-808c-250886e56780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f036ea7c0e94bb291d680aee2dae16d, user_id=5dda3c6f56c14b2ea40e594664dbfd2c, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.602684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57f743-fdea-43b6-808c-250886e56780 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.655630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6bada402a3433ab952142a066edfcf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.664299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6bada402a3433ab952142a066edfcf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.675760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1125/2249] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:59 2022] PUT /identity/v3/projects/0f036ea7c0e94bb291d680aee2dae16d/users/5dda3c6f56c14b2ea40e594664dbfd2c/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:59.679464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.679464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.679464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.701331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:59.702704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.769290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1126/2250] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:05:59.770110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.771839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6bada402a3433ab952142a066edfcf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.772892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6bada402a3433ab952142a066edfcf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.781793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eadd7-64cb-4731-b634-94eef7fe004f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.787106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6bada402a3433ab952142a066edfcf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:05:59.787653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.788263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.788692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.788858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.789650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.791623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.795896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1125/2251] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:05:59 2022] DELETE /identity/v3/users/4e6bada402a3433ab952142a066edfcf => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:59.798930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.798930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.812351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1127/2252] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:05:59.814651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:05:59.816584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:05:59.840117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef36834-80ca-4a1b-8ac5-eb55e3f8250a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.912442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3c49c-bb6f-4001-8bba-6a63fb6dbf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:05:59.914058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c49c-bb6f-4001-8bba-6a63fb6dbf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.914556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c49c-bb6f-4001-8bba-6a63fb6dbf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.915048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c49c-bb6f-4001-8bba-6a63fb6dbf1e admin admin] PATH_INFO: `/v3/projects/0f036ea7c0e94bb291d680aee2dae16d/users/5dda3c6f56c14b2ea40e594664dbfd2c/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.918450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3c49c-bb6f-4001-8bba-6a63fb6dbf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f036ea7c0e94bb291d680aee2dae16d, user_id=5dda3c6f56c14b2ea40e594664dbfd2c, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:05:59.932573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3c49c-bb6f-4001-8bba-6a63fb6dbf1e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:05:59.935169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1128/2253] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:05:59 2022] PUT /identity/v3/projects/0f036ea7c0e94bb291d680aee2dae16d/users/5dda3c6f56c14b2ea40e594664dbfd2c/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:05:59.938907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a89aa2-fd59-40da-b99e-77213f477d4c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:05:59.946674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:05:59.947032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:05:59.947443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:05:59.994539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] MFA Rules not processed for user `5dda3c6f56c14b2ea40e594664dbfd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:05:59.997240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:00.020051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.020851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.021231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.021636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.022437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:00.022844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.025186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.060517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1126/2254] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:00.063477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.076498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4951cbe-789e-48ea-b3a1-22d2619b5f24 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.085394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.086019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.086317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.087909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.088609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.091064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.143486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1129/2255] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:00.154891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.156538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:00.234906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b48a-adaa-4e97-950e-a39f8a45d918 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.239482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.240237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.240645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.241020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.253560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.255209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.268121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1127/2256] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:00 2022] POST /identity/v3/projects => generated 363 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:00.272475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.275625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1130/2257] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:00.358741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cdf30-9b14-4a90-8da6-c568c5acae30 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.363921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.375214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.375214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.375214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.375214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.375214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.436880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.439164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:00.468200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc4978-d49e-403c-b1c1-8888b7cb0872 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.481324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.482094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.482374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.482649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] PATH_INFO: `/v3/projects/e4a54187fe114b489867b336e66ae646` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.483807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a54187fe114b489867b336e66ae646)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.485792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.508361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1128/2258] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:00 2022] POST /identity/v3/users => generated 282 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:00.509794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.521357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7789347-6a13-4b5b-8487-a3792d80f94a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.531008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.531623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.531900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.532184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.533130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.534938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.549571 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1129/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:00.550799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.561681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7eac7-4bec-43b4-b635-69b0f3b359fe admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.569846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.570668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.570945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.571214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.571861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.574042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.589223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1130/2260] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:00.594099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.605098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c70530-f116-43b7-a785-c2f7c5f5a292 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:00.609990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dab99-0b65-45fb-b447-d88c0749096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:00.611352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dab99-0b65-45fb-b447-d88c0749096b admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.611606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dab99-0b65-45fb-b447-d88c0749096b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.611874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dab99-0b65-45fb-b447-d88c0749096b admin admin] PATH_INFO: `/v3/projects/e77ce4d150464e518e684f083d9b28e8/users/ce6179c2a2dd404f933d0779aa7d7610/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.628885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dab99-0b65-45fb-b447-d88c0749096b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77ce4d150464e518e684f083d9b28e8, user_id=ce6179c2a2dd404f933d0779aa7d7610, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:00.631948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dab99-0b65-45fb-b447-d88c0749096b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:00.796918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1131/2261] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:00 2022] PUT /identity/v3/projects/e77ce4d150464e518e684f083d9b28e8/users/ce6179c2a2dd404f933d0779aa7d7610/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:00.807780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a54187fe114b489867b336e66ae646', 'request_id': 'req-12e2d19d-735b-49f8-acca-388e4b93d8ae'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:00.813511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.814351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.814622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:00.844730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] MFA Rules not processed for user `ce6179c2a2dd404f933d0779aa7d7610`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:00.846159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:00.923796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1132/2262] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:00.928038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:00.940444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a54187fe114b489867b336e66ae646', 'request_id': 'req-12e2d19d-735b-49f8-acca-388e4b93d8ae'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:00.953455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a54187fe114b489867b336e66ae646', 'request_id': 'req-12e2d19d-735b-49f8-acca-388e4b93d8ae'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:00.971355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] The token cache is being invalidate because project e4a54187fe114b489867b336e66ae646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:00.971737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:00.972543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-12e2d19d-735b-49f8-acca-388e4b93d8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:00.976712 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1131/2263] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:00 2022] DELETE /identity/v3/projects/e4a54187fe114b489867b336e66ae646 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:00.979215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7344c62-1a2a-4933-b2b8-e81e26a86088 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:00.982417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7344c62-1a2a-4933-b2b8-e81e26a86088 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:00.982417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7344c62-1a2a-4933-b2b8-e81e26a86088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.012246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e7344c62-1a2a-4933-b2b8-e81e26a86088 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:01.012246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e7344c62-1a2a-4933-b2b8-e81e26a86088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:01.013010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c19c4-4d0b-47c5-8e47-79e1af147e8b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.016348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.017748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.018025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.018332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.018977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.021071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1132/2264] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:01.021361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.030843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.031205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.031553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.053819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:01.062823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:01.063695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1133/2265] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:01.068043 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.076852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06b39-d5c0-4cb5-b58a-93c9f55fe3ce None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.083924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.121525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.121525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.121525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.121525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:01.121525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.121525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.122375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1134/2266] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:06:01.126409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.137612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe012a80-c7c4-4263-8098-93640b27309c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.147233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.147963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.148355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.160453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.160453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.160453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.195761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1133/2267] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:00 2022] POST /identity/v3/projects => generated 365 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:01.201046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.213229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:01.229585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:01.311681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deb6c9-4010-447c-8cef-3d5d7437f170 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.317124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.317743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.318028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.318307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.318958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.320951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.523622 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1135/2268] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:01 2022] POST /identity/v3/projects => generated 385 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:01.524098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.593878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccfabe-7e24-4016-b90d-0799b439e3f0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.599244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.601305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.601305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.601305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.601305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.603229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.639957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1134/2269] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:01 2022] POST /identity/v3/users => generated 289 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:01.643660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.655812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c2e6-3357-41e2-8150-6e9b20a6ca95 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.662082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.662724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.663009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.663287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.663972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.665665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.679553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1135/2270] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:01.684820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.696401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048abfa-7896-402d-9c43-29a383bd4904 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.702671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.703361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.703645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.703922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] PATH_INFO: `/v3/projects/b5f74a3672ea45f7a1c971ae27b8c6f6/users/0cf23cb4478346a9bea9edf39b8cada5/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.720661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f74a3672ea45f7a1c971ae27b8c6f6, user_id=0cf23cb4478346a9bea9edf39b8cada5, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.723338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.787344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1136/2271] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:01 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:01.791624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.803050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc14c4-3c0f-47e6-a23f-d67c1fd30a53 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.808885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.814532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.814532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.814532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.814532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.814532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.824117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1137/2272] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:01.827720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.841198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc857c7-d081-4b87-abf0-dc45f53e1f8f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.847056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.847782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.848219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.848628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.849376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.851211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.869298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1136/2273] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:01 2022] PUT /identity/v3/projects/b5f74a3672ea45f7a1c971ae27b8c6f6/users/0cf23cb4478346a9bea9edf39b8cada5/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:01.869779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1138/2274] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:01.873634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.875699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:01.892098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:01.901095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd710e18-6b80-4c44-96c3-b241d243fba9 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.948578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890774cf-27f4-425f-987d-3354693a23df admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:01.953527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2247b8d-ed1e-40dc-9618-dc65db106ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.954240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2247b8d-ed1e-40dc-9618-dc65db106ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.954582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2247b8d-ed1e-40dc-9618-dc65db106ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.954875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2247b8d-ed1e-40dc-9618-dc65db106ca1 admin admin] PATH_INFO: `/v3/projects/b4050c5490a947529d4589e967d2851f/users/96522efdc0cf49e59f69d98cf45f0b88/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.971715 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:01.972371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:01.972655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:01.972943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:01.973590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.975229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.984975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2247b8d-ed1e-40dc-9618-dc65db106ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4050c5490a947529d4589e967d2851f, user_id=96522efdc0cf49e59f69d98cf45f0b88, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:01.986791 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1139/2275] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:01.988685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2247b8d-ed1e-40dc-9618-dc65db106ca1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:01.990747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:02.002291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3c953-33b5-4f89-9080-26185c9b95b4 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:02.007062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17369f-ebd6-4f41-81dd-ca6091efa722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:02.007717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17369f-ebd6-4f41-81dd-ca6091efa722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:02.007998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17369f-ebd6-4f41-81dd-ca6091efa722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:02.008276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17369f-ebd6-4f41-81dd-ca6091efa722 admin admin] PATH_INFO: `/v3/projects/b5f74a3672ea45f7a1c971ae27b8c6f6/users/0cf23cb4478346a9bea9edf39b8cada5/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:02.010317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17369f-ebd6-4f41-81dd-ca6091efa722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f74a3672ea45f7a1c971ae27b8c6f6, user_id=0cf23cb4478346a9bea9edf39b8cada5, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:02.012965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17369f-ebd6-4f41-81dd-ca6091efa722 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:02.023640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1140/2276] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:01 2022] PUT /identity/v3/projects/b5f74a3672ea45f7a1c971ae27b8c6f6/users/0cf23cb4478346a9bea9edf39b8cada5/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:02.034995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:02.035324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:02.035601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:02.067145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] MFA Rules not processed for user `0cf23cb4478346a9bea9edf39b8cada5`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:02.067145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:02.108612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1137/2277] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:01 2022] PUT /identity/v3/projects/b4050c5490a947529d4589e967d2851f/users/96522efdc0cf49e59f69d98cf45f0b88/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:02.110310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:02.110680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:02.111033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:02.144257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1141/2278] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:02.147867 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] MFA Rules not processed for user `96522efdc0cf49e59f69d98cf45f0b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:02.149113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:02.193559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:02.195355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:02.219082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1138/2279] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:02.241491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:02.243403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:02.269925 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d066d0-97a6-4da4-b071-751e70b93843 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:02.305011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29aa36-be18-4d3b-8222-12685acadfee None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:02.309916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:02.310541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:02.310825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:02.311100 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:02.312981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:02.326374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:02.329221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:02.329221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:02.329221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:02.377512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:02.379123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:02.392472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1139/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:02.403935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:02.403935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:02.411385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1142/2281] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:02.546174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:02.562402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0afd2b-6a90-4a0c-b4f4-746ae6e186a9 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:02.567342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54aa62-8870-469c-97ff-5ba21d768043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:02.567962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54aa62-8870-469c-97ff-5ba21d768043 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:02.568245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54aa62-8870-469c-97ff-5ba21d768043 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:02.578312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54aa62-8870-469c-97ff-5ba21d768043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:02.578312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54aa62-8870-469c-97ff-5ba21d768043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:02.578580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54aa62-8870-469c-97ff-5ba21d768043 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:02.587546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1140/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:03.063793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:04.064141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:05.140474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:05.204562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae713b3-eab3-4601-b6b6-1e13166d5972 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:05.209470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:05.210223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:05.210486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:05.210765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] PATH_INFO: `/v3/users/ce6179c2a2dd404f933d0779aa7d7610` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:05.211960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6179c2a2dd404f933d0779aa7d7610)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:05.213943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:05.444905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6179c2a2dd404f933d0779aa7d7610'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:05.451721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6179c2a2dd404f933d0779aa7d7610'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:05.452465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6179c2a2dd404f933d0779aa7d7610'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:05.453084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6179c2a2dd404f933d0779aa7d7610'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:05.577136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6179c2a2dd404f933d0779aa7d7610'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:05.595563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1143/2283] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:05 2022] DELETE /identity/v3/users/ce6179c2a2dd404f933d0779aa7d7610 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:05.600175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313d830-1365-4b5f-9406-949f266b182d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:05.600655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313d830-1365-4b5f-9406-949f266b182d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:05.600849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313d830-1365-4b5f-9406-949f266b182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:05.622866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f313d830-1365-4b5f-9406-949f266b182d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:05.624257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f313d830-1365-4b5f-9406-949f266b182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:05.692709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1141/2284] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:05.702570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:05.706339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:05.765730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18f693-1414-49f6-8197-63e85d083731 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:05.769726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:05.770355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:05.770633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:05.770914 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:05.780298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:05.781751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:05.792269 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1144/2285] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:06:05.983906 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:06.435182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:06.717051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313d830-1365-4b5f-9406-949f266b182d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:06.718676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f313d830-1365-4b5f-9406-949f266b182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:06.731574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313d830-1365-4b5f-9406-949f266b182d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:06.736931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:06.737578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:06.737858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:06.738133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] PATH_INFO: `/v3/projects/e77ce4d150464e518e684f083d9b28e8` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:06.739282 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77ce4d150464e518e684f083d9b28e8)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:06.741284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:06.918058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77ce4d150464e518e684f083d9b28e8', 'request_id': 'req-1112073e-288a-4355-9fcd-955fc2ca5548'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:06.980448 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77ce4d150464e518e684f083d9b28e8', 'request_id': 'req-1112073e-288a-4355-9fcd-955fc2ca5548'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:06.986921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77ce4d150464e518e684f083d9b28e8', 'request_id': 'req-1112073e-288a-4355-9fcd-955fc2ca5548'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:06.998539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] The token cache is being invalidate because project e77ce4d150464e518e684f083d9b28e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:06.998836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:06.999473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.001264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1142/2286] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:06 2022] DELETE /identity/v3/projects/e77ce4d150464e518e684f083d9b28e8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:07.005500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:07.015257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcac9f-ac6d-4af0-970a-6f140e06db77 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:07.083334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:07.083988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:07.084278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:07.084588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] PATH_INFO: `/v3/users/0cf23cb4478346a9bea9edf39b8cada5` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:07.085771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf23cb4478346a9bea9edf39b8cada5)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:07.087755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:07.349845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf23cb4478346a9bea9edf39b8cada5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.356639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf23cb4478346a9bea9edf39b8cada5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.357300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf23cb4478346a9bea9edf39b8cada5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.357904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf23cb4478346a9bea9edf39b8cada5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.423483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf23cb4478346a9bea9edf39b8cada5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.442022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1145/2287] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:07 2022] DELETE /identity/v3/users/0cf23cb4478346a9bea9edf39b8cada5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:07.609288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:07.611682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:07.628616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112073e-288a-4355-9fcd-955fc2ca5548 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:07.714061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:07.714667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:07.715008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:07.715348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] PATH_INFO: `/v3/users/96522efdc0cf49e59f69d98cf45f0b88` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:07.716825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96522efdc0cf49e59f69d98cf45f0b88)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:07.719568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:07.889736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:07.979809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:07.981864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96522efdc0cf49e59f69d98cf45f0b88'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:07.982178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:07.989923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96522efdc0cf49e59f69d98cf45f0b88'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.001942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4e34c-42c0-44d8-9b7b-0f7ae46dc94e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:08.008137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:08.008941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.009219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.009493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] PATH_INFO: `/v3/projects/b5f74a3672ea45f7a1c971ae27b8c6f6` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.010803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f74a3672ea45f7a1c971ae27b8c6f6)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:08.013632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:08.080788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96522efdc0cf49e59f69d98cf45f0b88'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.081472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96522efdc0cf49e59f69d98cf45f0b88'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.094221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7e6586bf-17fd-468b-a4cc-ca9ff46e9904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96522efdc0cf49e59f69d98cf45f0b88'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.101031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1143/2288] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:07 2022] DELETE /identity/v3/users/96522efdc0cf49e59f69d98cf45f0b88 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:08.105378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.105664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.105946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.126204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:08.127594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:08.145628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f74a3672ea45f7a1c971ae27b8c6f6', 'request_id': 'req-7c400a93-f7a5-48ac-a601-bfe27d37690a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.194992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1144/2289] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:08.204727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:08.206080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:08.241737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f74a3672ea45f7a1c971ae27b8c6f6', 'request_id': 'req-7c400a93-f7a5-48ac-a601-bfe27d37690a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.249828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f74a3672ea45f7a1c971ae27b8c6f6', 'request_id': 'req-7c400a93-f7a5-48ac-a601-bfe27d37690a'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.269959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] The token cache is being invalidate because project b5f74a3672ea45f7a1c971ae27b8c6f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:08.270130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cf43c-9d9f-48f3-8b06-f8da7a5efda8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:08.270257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.271030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7c400a93-f7a5-48ac-a601-bfe27d37690a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:08.272762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1146/2290] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:07 2022] DELETE /identity/v3/projects/b5f74a3672ea45f7a1c971ae27b8c6f6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:08.277526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf30455-001a-4573-8934-5d7bebe6bd0e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.277855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf30455-001a-4573-8934-5d7bebe6bd0e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.283477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf30455-001a-4573-8934-5d7bebe6bd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.299151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4bf30455-001a-4573-8934-5d7bebe6bd0e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:08.300396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4bf30455-001a-4573-8934-5d7bebe6bd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:08.326721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:08.327355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.327631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.327906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.330306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:08.370861 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1147/2291] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:08.378596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.381041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.381041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.400560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:08.401555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:08.401763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:08.403331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:08.411719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1148/2292] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:08.415993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:08.416375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1145/2293] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:06:08.425554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd452dc7-e6a5-487d-9f16-6dcc432c87a2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:08.430332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:08.430937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.431268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.431562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.432197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:08.432519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:08.434077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:08.447000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1149/2294] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:08.452483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:08.462815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b5e7d-4ddb-4e71-bcad-eb924395a158 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:08.467813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:08.468521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.468802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.469126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.470476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:08.472578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:08.678918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1146/2295] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:08 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:08.680134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:08.691811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:08.693690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:08.703702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18717c31-ce88-4058-b6a4-40a64fe163b6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:08.779751 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0749c9-2da5-48cd-96b1-afef98e57d9d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:08.786362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:08.787009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.787423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.787834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] PATH_INFO: `/v3/users/06c3e87487c34845bfb24ceccf0aff7c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.789028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c3e87487c34845bfb24ceccf0aff7c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:08.790960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:08.791283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:08.791998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:08.792282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:08.792583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:08.793543 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:08.796286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.032241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1150/2296] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:08 2022] POST /identity/v3/users => generated 282 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:09.036603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.046461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff313-de4e-4072-9080-0fb34d66a9c7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.051878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.052503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.052783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.053056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.053689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.055247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.073753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1151/2297] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:09.077594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.079398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.100684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffac78-c7bb-4fb2-ac09-f0bdc6742910 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.106349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.107528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.107528 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.107736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] PATH_INFO: `/v3/projects/b4050c5490a947529d4589e967d2851f` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.108892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4050c5490a947529d4589e967d2851f)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.111587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.166242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c3e87487c34845bfb24ceccf0aff7c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.173911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c3e87487c34845bfb24ceccf0aff7c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.241895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4050c5490a947529d4589e967d2851f', 'request_id': 'req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.258587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c3e87487c34845bfb24ceccf0aff7c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.259210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c3e87487c34845bfb24ceccf0aff7c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.272845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c3e87487c34845bfb24ceccf0aff7c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.279049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1147/2298] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:08 2022] DELETE /identity/v3/users/06c3e87487c34845bfb24ceccf0aff7c => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:09.279778 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.281145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.337510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4050c5490a947529d4589e967d2851f', 'request_id': 'req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.343986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4050c5490a947529d4589e967d2851f', 'request_id': 'req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.349858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772fe9-b175-4ed0-80f9-1610d2076326 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.354727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a598a-b406-444b-8a9e-b2b36109b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.355334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a598a-b406-444b-8a9e-b2b36109b2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.355619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a598a-b406-444b-8a9e-b2b36109b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.355745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] The token cache is being invalidate because project b4050c5490a947529d4589e967d2851f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:09.355990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.356123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a598a-b406-444b-8a9e-b2b36109b2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.356666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a598a-b406-444b-8a9e-b2b36109b2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.356774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-1b3f2fad-0853-49d7-b9d2-44d5cd53bb30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:09.358164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1152/2299] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:09 2022] DELETE /identity/v3/projects/b4050c5490a947529d4589e967d2851f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:09.358301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a598a-b406-444b-8a9e-b2b36109b2b8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.359549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.359787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.360873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.372330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1148/2300] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:09.375554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.375554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.375554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.380366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:09.381571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.393020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1153/2301] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:09.393020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.393020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.394059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:09.395239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.404695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d6e55-898f-4921-88dd-bed1210d8893 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.411696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1997d8-87fb-4b9a-88ad-fd70d1131be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.412426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1997d8-87fb-4b9a-88ad-fd70d1131be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.412770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1997d8-87fb-4b9a-88ad-fd70d1131be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.412974 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1997d8-87fb-4b9a-88ad-fd70d1131be0 admin admin] PATH_INFO: `/v3/projects/12aa7e10c4b74de18431a60befaa324c/users/5434fbd6018346bf8ce0fd150b490dd0/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.416698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1149/2302] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:09.417684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.419059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.433285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1997d8-87fb-4b9a-88ad-fd70d1131be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12aa7e10c4b74de18431a60befaa324c, user_id=5434fbd6018346bf8ce0fd150b490dd0, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.436335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1997d8-87fb-4b9a-88ad-fd70d1131be0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.458126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:09.480801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f264492-dcee-4072-afe8-ad5f7606cc0b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.484627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412539c9-7f33-426c-9d26-74c54138e96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.485620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412539c9-7f33-426c-9d26-74c54138e96a service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.485908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412539c9-7f33-426c-9d26-74c54138e96a service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.486184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412539c9-7f33-426c-9d26-74c54138e96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.503665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412539c9-7f33-426c-9d26-74c54138e96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.505208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412539c9-7f33-426c-9d26-74c54138e96a service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.515420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1150/2303] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:09.516951 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.517799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.518247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.519297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1154/2304] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:09 2022] PUT /identity/v3/projects/12aa7e10c4b74de18431a60befaa324c/users/5434fbd6018346bf8ce0fd150b490dd0/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:09.529802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.530218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.530596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.540969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:09.542196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.572131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] MFA Rules not processed for user `5434fbd6018346bf8ce0fd150b490dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:09.584606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:09.633061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1151/2305] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:09.637313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.649157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f66e26-7b0e-4ca7-a426-2bf252c36a40 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.654362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.655013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.655288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.655687 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.656470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:09.656777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.659166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.672404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1155/2306] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:09.680675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1152/2307] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:09.681831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.681958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.692917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b4c2e-bb96-411c-8b58-b486b00d94ac admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.696652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f1f27-a6b7-47ce-ad6e-b63ed152fef5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:09.699229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.699420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.699717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.699982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.700705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.702114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:09.702654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.702935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:09.703326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:09.703729 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:09.704536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:09.706485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:09.845158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1153/2308] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:09.846698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.931075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1156/2309] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/projects => generated 365 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:09.932224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:09.934162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f187a-b108-4c2b-9268-70fb944fba68 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.035119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.035888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.036475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.036827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.038080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.040484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.060409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60bb2-6750-457e-b2d5-a3f8f6e5fa30 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.085758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.085758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.085758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.085758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.086461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.086461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.420602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1157/2310] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/users => generated 289 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:10.425335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.426969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:10.438671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52550241-92c5-4cab-a725-5a51e090cdf1 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.443144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.443824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.444107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.444420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] PATH_INFO: `/v3/projects/8c5fa871ae8a46cbac26cc1b8646f4ba` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.445594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5fa871ae8a46cbac26cc1b8646f4ba)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.447458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.464868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1154/2311] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:09 2022] POST /identity/v3/users => generated 287 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:10.467330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.473475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:10.532820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570aff1d-7a03-4039-addb-e902359e2570 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.538380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.539129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.539399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.539703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.541820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:10.604633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.606010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.615391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1155/2312] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:10.625297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.628738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f7c16-8b3f-45d5-9d24-17b796c47a04 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.638141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5fa871ae8a46cbac26cc1b8646f4ba', 'request_id': 'req-4953d92c-6244-4a3b-b94a-2fc699bf1351'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:10.696838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.699923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.699923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.699923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.699923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.700786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.715123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1156/2313] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:10.716255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.727771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be6132-7c80-41a8-b76e-7dddffca61b4 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.732640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.733252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.733532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.733805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.734459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.736185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.746195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1157/2314] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:10.746964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.758348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e302-7ba3-41eb-8b61-4673fed50169 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.762955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f564a-6970-4912-a1b6-21890f3dc3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.763581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f564a-6970-4912-a1b6-21890f3dc3d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.763864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f564a-6970-4912-a1b6-21890f3dc3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.764137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f564a-6970-4912-a1b6-21890f3dc3d2 admin admin] PATH_INFO: `/v3/projects/063c33023e6b431fbf4b108bc54152bc/users/1546f49729794a68b8960b35477648f3/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.778752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f564a-6970-4912-a1b6-21890f3dc3d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063c33023e6b431fbf4b108bc54152bc, user_id=1546f49729794a68b8960b35477648f3, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.780391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5fa871ae8a46cbac26cc1b8646f4ba', 'request_id': 'req-4953d92c-6244-4a3b-b94a-2fc699bf1351'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:10.781385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f564a-6970-4912-a1b6-21890f3dc3d2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.787644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5fa871ae8a46cbac26cc1b8646f4ba', 'request_id': 'req-4953d92c-6244-4a3b-b94a-2fc699bf1351'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:10.799496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] The token cache is being invalidate because project 8c5fa871ae8a46cbac26cc1b8646f4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:10.799758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:10.800458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:10.801940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1158/2315] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:10 2022] DELETE /identity/v3/projects/8c5fa871ae8a46cbac26cc1b8646f4ba => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:10.805260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.805260 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:10.816879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d92c-6244-4a3b-b94a-2fc699bf1351 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.822512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b057b4-2596-4f4d-ab7d-699197b47225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.826120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b057b4-2596-4f4d-ab7d-699197b47225 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.826120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b057b4-2596-4f4d-ab7d-699197b47225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.826120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b057b4-2596-4f4d-ab7d-699197b47225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.826120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b057b4-2596-4f4d-ab7d-699197b47225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.826120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b057b4-2596-4f4d-ab7d-699197b47225 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.840351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1159/2316] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:10.840351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.840351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.840351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.859570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:10.860965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:10.868930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1160/2317] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:10.869707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:10.878985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8686-c2fe-4774-ac99-ab2137649dc1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:10.883566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:10.884202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.884525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.884836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] PATH_INFO: `/v3/projects/0c9396351877403e8292c289dab90549/users/20239fe662f94c3ba25d3560a8c5ac3d/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.899965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9396351877403e8292c289dab90549, user_id=20239fe662f94c3ba25d3560a8c5ac3d, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:10.902707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:10.953843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1158/2318] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:10 2022] PUT /identity/v3/projects/063c33023e6b431fbf4b108bc54152bc/users/1546f49729794a68b8960b35477648f3/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:10.953843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d8695-8067-4d48-a044-bc8596ebde94 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:10.953843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d8695-8067-4d48-a044-bc8596ebde94 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:10.953843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d8695-8067-4d48-a044-bc8596ebde94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:10.974499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1e4d8695-8067-4d48-a044-bc8596ebde94 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:10.975729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1e4d8695-8067-4d48-a044-bc8596ebde94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:11.037541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1159/2319] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:11.038952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.039248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.039525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.058037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1161/2320] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:10 2022] PUT /identity/v3/projects/0c9396351877403e8292c289dab90549/users/20239fe662f94c3ba25d3560a8c5ac3d/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:11.058491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] MFA Rules not processed for user `01f1499bbf89441dae30433aaf1d29a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:11.058950 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.059712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:11.060709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:11.119702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1160/2321] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:11.120670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.134501 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34a30-933a-4295-8d29-1b5c33e3ebce admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.140468 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cc50e-5576-417e-8bdf-d9f6a28d9012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.142984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cc50e-5576-417e-8bdf-d9f6a28d9012 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.142984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cc50e-5576-417e-8bdf-d9f6a28d9012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.142984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cc50e-5576-417e-8bdf-d9f6a28d9012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.143333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cc50e-5576-417e-8bdf-d9f6a28d9012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.144825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cc50e-5576-417e-8bdf-d9f6a28d9012 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.155226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1162/2322] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:11.156576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220008b2-f68c-441f-9f9d-836b36527681 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.156940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220008b2-f68c-441f-9f9d-836b36527681 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.157380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220008b2-f68c-441f-9f9d-836b36527681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.195118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db764-89ca-4e54-b88b-3ddcbf376dfc None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.196986 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-220008b2-f68c-441f-9f9d-836b36527681 None None] MFA Rules not processed for user `20239fe662f94c3ba25d3560a8c5ac3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:11.198267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-220008b2-f68c-441f-9f9d-836b36527681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:11.200934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.264493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.264493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.264493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.264493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:11.264493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.264493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.265603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1161/2323] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:11.265603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.265603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc6b6c-0369-4e69-a09f-832b1a4c4070 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.265603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.265958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.265958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.265958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] PATH_INFO: `/v3/projects/063c33023e6b431fbf4b108bc54152bc/users/1546f49729794a68b8960b35477648f3/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.265958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063c33023e6b431fbf4b108bc54152bc, user_id=1546f49729794a68b8960b35477648f3, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.265958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.267078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1162/2324] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:11 2022] PUT /identity/v3/projects/063c33023e6b431fbf4b108bc54152bc/users/1546f49729794a68b8960b35477648f3/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:11.269359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.287877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1163/2325] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:11.289416 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.289699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.300379 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.320709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] MFA Rules not processed for user `1546f49729794a68b8960b35477648f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:11.322022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:11.343735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a48ea4-49f1-4b0f-8e6c-ed7e5cd775a5 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.351993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.351993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.351993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.351993 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.352607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.353639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.402733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1164/2326] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:11.402733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.402733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:11.460258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66d657-ecea-4625-957a-0e3a23d81ff1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.464093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.464824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.465117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.465386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.523099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.524743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.535291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1165/2327] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:11.574032 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1163/2328] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:11 2022] POST /identity/v3/projects => generated 391 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:11.574328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.574626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.587742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bccff-6a10-4b86-945d-d1de246f8678 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.642807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8975f-e9e1-4828-b2d0-0fb98cc3e6bb admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.648304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.649176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.649331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.649593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.649795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.649917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] PATH_INFO: `/v3/users/20239fe662f94c3ba25d3560a8c5ac3d` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.650090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.650405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.651092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.651202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] RBAC: Authorizing `identity:delete_user(user_id=20239fe662f94c3ba25d3560a8c5ac3d)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.652969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.653090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.855002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1166/2329] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:11 2022] POST /identity/v3/users => generated 296 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:11.855854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:11.913201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03354e-2edb-4401-8c2c-60b0f6429573 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:11.913420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:11.914002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:11.914279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:11.914548 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:11.935555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:11.979505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:11.981048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:11.991710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1167/2330] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:11.992689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:12.008448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a9cd3-4c97-453f-929b-a0d337f96a4e service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:12.014460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:12.015191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.015438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.015714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.017018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:12.018628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:12.031310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1168/2331] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:12.035894 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:12.046480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f61fdb-a918-4f19-a6cd-1dbaa935f1e0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:12.052095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:12.053017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.053470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.053898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.054851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:12.056701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:12.070213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1169/2332] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:12.074372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:12.085105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b11993-ade3-44d4-8f71-75f656760573 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:12.090040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44654f-bfd9-469a-a35a-ddf21f56f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:12.090699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44654f-bfd9-469a-a35a-ddf21f56f841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.090975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44654f-bfd9-469a-a35a-ddf21f56f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.091248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44654f-bfd9-469a-a35a-ddf21f56f841 admin admin] PATH_INFO: `/v3/projects/beafb1d075f047a38091a01899dc4ca7/users/50dfe6052ebe493dba087b81d34bc380/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.106566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44654f-bfd9-469a-a35a-ddf21f56f841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beafb1d075f047a38091a01899dc4ca7, user_id=50dfe6052ebe493dba087b81d34bc380, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:12.109237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44654f-bfd9-469a-a35a-ddf21f56f841 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:12.203311 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20239fe662f94c3ba25d3560a8c5ac3d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:12.210528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20239fe662f94c3ba25d3560a8c5ac3d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:12.241740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1170/2333] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:12 2022] PUT /identity/v3/projects/beafb1d075f047a38091a01899dc4ca7/users/50dfe6052ebe493dba087b81d34bc380/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:12.253124 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b2f4-6419-489e-96b1-e96e71384184 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.253440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b2f4-6419-489e-96b1-e96e71384184 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.253727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3b2f4-6419-489e-96b1-e96e71384184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.282891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-aea3b2f4-6419-489e-96b1-e96e71384184 None None] MFA Rules not processed for user `50dfe6052ebe493dba087b81d34bc380`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:12.284410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-aea3b2f4-6419-489e-96b1-e96e71384184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.338145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20239fe662f94c3ba25d3560a8c5ac3d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:12.339732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:12.340568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20239fe662f94c3ba25d3560a8c5ac3d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:12.355272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1171/2334] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:12.355461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20239fe662f94c3ba25d3560a8c5ac3d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:12.362086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1164/2335] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:11 2022] DELETE /identity/v3/users/20239fe662f94c3ba25d3560a8c5ac3d => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:12.366374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.366667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.366944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.380403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:12.381900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.389705 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:12.390921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.440452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8689c89-74a3-48fe-b44c-93875378430c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:12.444962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28001405-1599-4b01-b106-5836083465db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:12.445613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28001405-1599-4b01-b106-5836083465db service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.445888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28001405-1599-4b01-b106-5836083465db service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.446216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28001405-1599-4b01-b106-5836083465db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.449458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-28001405-1599-4b01-b106-5836083465db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.450525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1172/2336] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:12.462400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:12.463291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.523364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28001405-1599-4b01-b106-5836083465db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:12.524874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28001405-1599-4b01-b106-5836083465db service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:12.530131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662360-44ac-4dbc-811c-cdd6b024e308 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:12.534625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1165/2337] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:06:12.542098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:12.542098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.542098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.542098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.552044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:12.553499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:12.563039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1173/2338] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:12.807446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28001405-1599-4b01-b106-5836083465db service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:12.808871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-28001405-1599-4b01-b106-5836083465db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.821376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28001405-1599-4b01-b106-5836083465db service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:12.824457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:12.825218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:12.825522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:12.825910 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:12.828003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:12.902305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:12.903920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711011-2e67-41fa-980c-760067ef5665 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:12.915145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1166/2339] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:13.079554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:13.081819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:13.096436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaf71b-ca02-4dc9-bb98-c905face4720 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:13.116613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:13.117364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:13.117968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:13.118128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] PATH_INFO: `/v3/projects/0c9396351877403e8292c289dab90549` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:13.128875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9396351877403e8292c289dab90549)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:13.128875 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:13.333514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9396351877403e8292c289dab90549', 'request_id': 'req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:13.460770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9396351877403e8292c289dab90549', 'request_id': 'req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:13.468163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9396351877403e8292c289dab90549', 'request_id': 'req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:13.503288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] The token cache is being invalidate because project 0c9396351877403e8292c289dab90549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:13.503671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:13.504492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-69ff2d75-d8cf-4948-ac4c-117d6707ebc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:13.510761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1174/2340] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:13 2022] DELETE /identity/v3/projects/0c9396351877403e8292c289dab90549 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:13.514422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:13.514912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:13.515110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:13.537106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:13.538061 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:13.599419 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1167/2341] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:13.611368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:13.611652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:13.614056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:13.645630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:13.646927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:13.666520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:13.666728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1175/2342] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:13.678695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed5689-400b-4986-bc0e-19df296aec2a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:13.683742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:13.684452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:13.684865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:13.685209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:13.686202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:13.687144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:13.705785 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:13.719098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1168/2343] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:13.723605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:13.733037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d85e9-7a74-4469-be00-f2fe0d67cf6f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:13.738333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:13.739022 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:13.739301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:13.739565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:13.740193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:13.742139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:13.879500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1176/2344] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:13 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:13.883928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:13.894402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e7ea0-1c64-4ecc-a045-57bdac5bdc98 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:13.953614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:13.954214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:13.954493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:13.954763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:13.955388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:13.957409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:14.126487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1169/2345] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:13 2022] POST /identity/v3/users => generated 292 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:14.131695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:14.141807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f13940-736c-4474-be8d-5b7f7d6eaa37 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:14.146505 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:14.147097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:14.147373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:14.147644 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:14.148281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:14.149908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:14.160924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1177/2346] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:14.164926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:14.175505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d75718-0d17-4b8b-bcf2-fa66fd3d235c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:14.180338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da95ba7-03bb-40b1-8009-0cb533da6872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:14.181030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95ba7-03bb-40b1-8009-0cb533da6872 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:14.181314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95ba7-03bb-40b1-8009-0cb533da6872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:14.181591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da95ba7-03bb-40b1-8009-0cb533da6872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:14.182247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da95ba7-03bb-40b1-8009-0cb533da6872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:14.183874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da95ba7-03bb-40b1-8009-0cb533da6872 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:14.199195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1170/2347] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:14.214021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:14.226286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d80db-ac0f-4173-bd56-d11a868be799 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:14.231346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:14.232139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:14.232556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:14.244837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] PATH_INFO: `/v3/projects/ab11fffacccb4f248d76d99553bdb2a9/users/6201e7d144874370b4da432a529014d2/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:14.271301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab11fffacccb4f248d76d99553bdb2a9, user_id=6201e7d144874370b4da432a529014d2, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:14.280881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:14.503665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1178/2348] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:14 2022] PUT /identity/v3/projects/ab11fffacccb4f248d76d99553bdb2a9/users/6201e7d144874370b4da432a529014d2/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:14.508814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2187a2-f5dd-46b6-83fe-99f9f76c6ef8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:14.509540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2187a2-f5dd-46b6-83fe-99f9f76c6ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:14.509540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2187a2-f5dd-46b6-83fe-99f9f76c6ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:14.532706 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-be2187a2-f5dd-46b6-83fe-99f9f76c6ef8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:14.534214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-be2187a2-f5dd-46b6-83fe-99f9f76c6ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:14.619078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1171/2349] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:14.629617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:14.632012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:14.702996 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d51bd-f6ee-4568-aed5-06e3f5925299 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:14.707908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:14.708605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:14.709000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:14.709236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:14.720247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:14.721789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:14.732286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1179/2350] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:15.376098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:16.303762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:18.085228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:18.085874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:18.085874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:18.115565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] MFA Rules not processed for user `6201e7d144874370b4da432a529014d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:18.116908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:18.182586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1172/2351] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:18.193566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:18.194944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:18.250783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c222b1-53ad-425d-b924-5896e06766b3 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:18.254613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46be77-e43b-4497-a23b-2e322714d0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:18.255225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46be77-e43b-4497-a23b-2e322714d0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:18.255502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46be77-e43b-4497-a23b-2e322714d0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:18.255837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46be77-e43b-4497-a23b-2e322714d0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:18.269449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46be77-e43b-4497-a23b-2e322714d0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:18.271074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46be77-e43b-4497-a23b-2e322714d0b5 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:18.280273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1180/2352] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:19.804835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:19.806276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:19.821013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13551eae-c5c2-41da-9a1b-3f75a24d0d25 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:19.826312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:19.826959 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:19.827242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:19.827519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] PATH_INFO: `/v3/users/7d4abb23a1d94b39b9b14422c9a2eeb7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:19.828697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4abb23a1d94b39b9b14422c9a2eeb7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:19.831757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:20.372190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4abb23a1d94b39b9b14422c9a2eeb7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:20.379529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4abb23a1d94b39b9b14422c9a2eeb7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:20.460436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4abb23a1d94b39b9b14422c9a2eeb7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:20.462103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4abb23a1d94b39b9b14422c9a2eeb7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:20.475498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:20.478178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4abb23a1d94b39b9b14422c9a2eeb7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:20.484877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1173/2353] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:19 2022] DELETE /identity/v3/users/7d4abb23a1d94b39b9b14422c9a2eeb7 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:20.489434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:20.489924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:20.490128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:20.509028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:20.510144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:20.572371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1181/2354] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:20.589386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:20.589386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:20.643331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc81f4d-64aa-4337-8655-cc263d407d17 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:20.646988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:20.647596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:20.647875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:20.648144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:20.657252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:20.658967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:20.670498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1174/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:20.866466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:21.199907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:21.201388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:21.213908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efa2fc-973f-4ffc-89a6-9541c8b91758 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:21.220110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:21.220889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:21.221072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:21.221356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] PATH_INFO: `/v3/projects/366b44bbf20340d7b1924ff1543a3638` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:21.222720 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=366b44bbf20340d7b1924ff1543a3638)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:21.237803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:21.372290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '366b44bbf20340d7b1924ff1543a3638', 'request_id': 'req-4069f0ad-97fe-4525-9ffa-95227619a0ae'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.477185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '366b44bbf20340d7b1924ff1543a3638', 'request_id': 'req-4069f0ad-97fe-4525-9ffa-95227619a0ae'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.483287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '366b44bbf20340d7b1924ff1543a3638', 'request_id': 'req-4069f0ad-97fe-4525-9ffa-95227619a0ae'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.495700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] The token cache is being invalidate because project 366b44bbf20340d7b1924ff1543a3638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:21.496191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.496696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.498315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1182/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:21 2022] DELETE /identity/v3/projects/366b44bbf20340d7b1924ff1543a3638 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:21.504084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:21.514635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da13fe-cf51-4910-8b5e-2c57718ba4c0 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:21.571513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:21.572141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:21.572463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:21.572743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] PATH_INFO: `/v3/users/9e62be781db44b89955da9d06ceafaf6` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:21.573901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e62be781db44b89955da9d06ceafaf6)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:21.575834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:21.801500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:21.803982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e62be781db44b89955da9d06ceafaf6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.810876 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e62be781db44b89955da9d06ceafaf6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.891512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e62be781db44b89955da9d06ceafaf6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.892730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e62be781db44b89955da9d06ceafaf6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.905198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e62be781db44b89955da9d06ceafaf6'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:21.912138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1175/2357] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:21 2022] DELETE /identity/v3/users/9e62be781db44b89955da9d06ceafaf6 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:22.433241 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:22.434690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:22.448952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069f0ad-97fe-4525-9ffa-95227619a0ae admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:22.507778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:22.508428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:22.508758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:22.509028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] PATH_INFO: `/v3/projects/40d0a7e809a04755b53274e3d7ecbfc1` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:22.510137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d0a7e809a04755b53274e3d7ecbfc1)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:22.512008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:22.644434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d0a7e809a04755b53274e3d7ecbfc1', 'request_id': 'req-28369655-1e52-42e3-ba3e-e8847df42de1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:22.711606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d0a7e809a04755b53274e3d7ecbfc1', 'request_id': 'req-28369655-1e52-42e3-ba3e-e8847df42de1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:22.718153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d0a7e809a04755b53274e3d7ecbfc1', 'request_id': 'req-28369655-1e52-42e3-ba3e-e8847df42de1'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:22.729361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] The token cache is being invalidate because project 40d0a7e809a04755b53274e3d7ecbfc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:22.729654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:22.730358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:22.731869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1183/2358] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:22 2022] DELETE /identity/v3/projects/40d0a7e809a04755b53274e3d7ecbfc1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:22.736809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:22.738273 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:22.751745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba4a17-7a36-4a09-a1ff-4d8105372391 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:22.806572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:22.807205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:22.807481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:22.807749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] PATH_INFO: `/v3/users/5dda3c6f56c14b2ea40e594664dbfd2c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:22.808892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dda3c6f56c14b2ea40e594664dbfd2c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:22.810789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:23.097189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dda3c6f56c14b2ea40e594664dbfd2c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:23.104145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dda3c6f56c14b2ea40e594664dbfd2c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:23.294992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dda3c6f56c14b2ea40e594664dbfd2c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:23.295622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dda3c6f56c14b2ea40e594664dbfd2c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:23.308528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8d7fe8f9-68e0-4658-be8a-f699b8b0bac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dda3c6f56c14b2ea40e594664dbfd2c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:23.319651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1176/2359] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:22 2022] DELETE /identity/v3/users/5dda3c6f56c14b2ea40e594664dbfd2c => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:23.349665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:23.773687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:23.775283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:23.789329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369655-1e52-42e3-ba3e-e8847df42de1 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:23.847305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:23.847938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:23.848220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:23.848527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] PATH_INFO: `/v3/projects/0f036ea7c0e94bb291d680aee2dae16d` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:23.849657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f036ea7c0e94bb291d680aee2dae16d)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:23.851564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:24.008530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f036ea7c0e94bb291d680aee2dae16d', 'request_id': 'req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:24.105817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f036ea7c0e94bb291d680aee2dae16d', 'request_id': 'req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:24.112130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f036ea7c0e94bb291d680aee2dae16d', 'request_id': 'req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:24.123164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] The token cache is being invalidate because project 0f036ea7c0e94bb291d680aee2dae16d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:24.123453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:24.124081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e4549460-a67b-44ee-8aa1-33fe18d1a6eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:24.125598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1184/2360] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:23 2022] DELETE /identity/v3/projects/0f036ea7c0e94bb291d680aee2dae16d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:24.132624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.133109 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.133202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.152846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:24.154034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:24.212442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1177/2361] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:24.220873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.221164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.221444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.242030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:24.243258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:24.251943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1185/2362] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:24.256330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:24.266449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538ab5f-9d5b-4cfa-a71c-f161e0efbd94 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:24.273651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:24.273651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.273651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.273651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.274012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:24.274012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:24.274856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:24.286532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1178/2363] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:24.290830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:24.300810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ebd59-c2fc-41bb-9ef7-6295ff8c56d5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:24.305462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:24.306063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.316789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.316789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.316789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:24.316789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:24.416680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1186/2364] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:24 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:24.420919 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:24.431843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902633c9-a2c2-484b-9313-172ae4e8b852 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:24.490741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:24.492075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.492075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.492075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.492574 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:24.494485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:24.676584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1179/2365] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:24 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:24.679140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:24.691163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf3df-5724-4f27-8dc7-ee691296302f admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:24.695888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:24.696699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.697112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.697382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.698028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:24.699809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:24.714711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1187/2366] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:24.714802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:24.725536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd585ed6-2208-4230-aeee-5feba86bcf0c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:24.730146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efa5a8-f271-44ad-839e-926612f5c202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:24.734038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa5a8-f271-44ad-839e-926612f5c202 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.734038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa5a8-f271-44ad-839e-926612f5c202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.734038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efa5a8-f271-44ad-839e-926612f5c202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.734038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efa5a8-f271-44ad-839e-926612f5c202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:24.734038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efa5a8-f271-44ad-839e-926612f5c202 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:24.745415 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1180/2367] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:24.750090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:24.759367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:24.761322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad16e7-ae39-4b3f-83a0-940a3c911a07 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:24.767988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:24.767988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:24.767988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:24.767988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] PATH_INFO: `/v3/projects/f74df4eea28c42f885b7484cf327fbb9/users/67877d8cb07741de8a288a6f097de783/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:24.784942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74df4eea28c42f885b7484cf327fbb9, user_id=67877d8cb07741de8a288a6f097de783, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:24.787890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:25.011855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1188/2368] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:24 2022] PUT /identity/v3/projects/f74df4eea28c42f885b7484cf327fbb9/users/67877d8cb07741de8a288a6f097de783/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:25.023649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:25.023938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:25.024210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:25.053598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] MFA Rules not processed for user `67877d8cb07741de8a288a6f097de783`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:25.054934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:25.121505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1181/2369] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:25.126615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:25.187845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d151f-48d9-41f7-8ee0-f37ebe568ff8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:25.193217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:25.193824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:25.194106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:25.194382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:25.195044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:25.196993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:25.289219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:25.385624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1189/2370] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:25 2022] POST /identity/v3/projects => generated 379 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:25.389820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:25.401015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a7954-dbbf-47ea-8b22-4cd6642e969f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:25.457504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:25.458108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:25.458385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:25.458653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:25.459281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:25.461216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:25.678160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1182/2371] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:25 2022] POST /identity/v3/users => generated 290 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:25.682140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:25.692810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ddc0c-583b-4fb1-bb99-48b25d3e8cac admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:25.697461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:25.698051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:25.698353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:25.698626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:25.699265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:25.700867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:25.711264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1190/2372] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:25.715125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:25.725474 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e06e2-0a46-442c-84b0-bbbb61b57a9f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:25.730080 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579394d-d474-4b1d-98ae-718247ebd6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:25.730702 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579394d-d474-4b1d-98ae-718247ebd6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:25.730981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579394d-d474-4b1d-98ae-718247ebd6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:25.731257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579394d-d474-4b1d-98ae-718247ebd6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:25.731909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579394d-d474-4b1d-98ae-718247ebd6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:25.733538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579394d-d474-4b1d-98ae-718247ebd6f8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:25.744363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1183/2373] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:25.748751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:25.759752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805868d1-601f-4271-9c13-507c296796d9 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:25.764374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:25.776926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:25.776926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:25.776926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] PATH_INFO: `/v3/projects/96409331e9ca437aaf71e4006f3b0242/users/5898a1397345433c92f4255caf464136/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:25.780202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=96409331e9ca437aaf71e4006f3b0242, user_id=5898a1397345433c92f4255caf464136, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:25.782920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:26.057867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1191/2374] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:25 2022] PUT /identity/v3/projects/96409331e9ca437aaf71e4006f3b0242/users/5898a1397345433c92f4255caf464136/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:26.069528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:26.069753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:26.070052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:26.099911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] MFA Rules not processed for user `5898a1397345433c92f4255caf464136`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:26.101175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:26.164348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1184/2375] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:26.169030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:26.229118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0f16c-7347-43f1-88f6-3503617e85bd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:26.233751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:26.234360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:26.234638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:26.234919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:26.236199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:26.238424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:26.448409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1192/2376] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:26 2022] POST /identity/v3/projects => generated 379 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:26.452522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:26.463937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608e187-2c92-4239-ac07-3781f67fc9b4 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:26.518490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:26.519155 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:26.519436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:26.519713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:26.520362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:26.522286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:26.787078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1185/2377] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:26 2022] POST /identity/v3/users => generated 296 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:26.788366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:26.799298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a73e4-17f7-4715-bedf-1d2bdb4e6ad3 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:26.804168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:26.804810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:26.805095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:26.805372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:26.806021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:26.807573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:26.822145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1193/2378] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:26.823301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:26.834086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d53faa-0482-47fe-9188-e85aa17c74dc admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:26.839904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:26.840678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:26.840964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:26.841240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] PATH_INFO: `/v3/projects/55b2fb37bb5a44fe8df7d407c7da7983/users/7b3c8b45e7f140919bf324686e34a562/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:26.856638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b2fb37bb5a44fe8df7d407c7da7983, user_id=7b3c8b45e7f140919bf324686e34a562, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:26.861627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:27.051901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1186/2379] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:26 2022] PUT /identity/v3/projects/55b2fb37bb5a44fe8df7d407c7da7983/users/7b3c8b45e7f140919bf324686e34a562/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:27.052139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:27.052139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9638518-59aa-4650-9b3d-2e1967df82b0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:27.106961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c679938-7512-46a5-9dbb-3994d63cedbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:27.107661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c679938-7512-46a5-9dbb-3994d63cedbe admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:27.108047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c679938-7512-46a5-9dbb-3994d63cedbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:27.108214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c679938-7512-46a5-9dbb-3994d63cedbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:27.108908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c679938-7512-46a5-9dbb-3994d63cedbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:27.111072 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c679938-7512-46a5-9dbb-3994d63cedbe admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:27.121648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1194/2380] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:27.125360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:27.136542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415e02a-c8db-4b14-8d8e-6203a0169ad9 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:27.141128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:27.141792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:27.142066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:27.142344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] PATH_INFO: `/v3/projects/55b2fb37bb5a44fe8df7d407c7da7983/users/7b3c8b45e7f140919bf324686e34a562/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:27.144280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b2fb37bb5a44fe8df7d407c7da7983, user_id=7b3c8b45e7f140919bf324686e34a562, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:27.146949 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:27.157764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1187/2381] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:27 2022] PUT /identity/v3/projects/55b2fb37bb5a44fe8df7d407c7da7983/users/7b3c8b45e7f140919bf324686e34a562/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:27.168448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:27.169054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:27.169350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:27.197963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] MFA Rules not processed for user `7b3c8b45e7f140919bf324686e34a562`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:27.199165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:27.297149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1195/2382] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:27.310391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:27.312297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:27.370875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a507fc-c4fc-4aae-84ff-2c68ef18d293 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:27.374600 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba221b-c505-4d1d-8866-97208e2624e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:27.375211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba221b-c505-4d1d-8866-97208e2624e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:27.375490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba221b-c505-4d1d-8866-97208e2624e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:27.375762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba221b-c505-4d1d-8866-97208e2624e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:27.388354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba221b-c505-4d1d-8866-97208e2624e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:27.389787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba221b-c505-4d1d-8866-97208e2624e6 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:27.401019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1188/2383] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:27.526299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:27.543289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:27.544753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:27.556849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8b42-f4c0-4019-9954-e7e5cf848b71 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:27.560359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51f72b-1ff1-46ad-9399-791ad097848e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:27.560965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51f72b-1ff1-46ad-9399-791ad097848e service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:27.561245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51f72b-1ff1-46ad-9399-791ad097848e service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:27.561519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51f72b-1ff1-46ad-9399-791ad097848e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:27.618682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51f72b-1ff1-46ad-9399-791ad097848e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:27.620108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51f72b-1ff1-46ad-9399-791ad097848e service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:27.629433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1196/2384] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:27.970947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:27.972171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:27.972171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:27.991016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] MFA Rules not processed for user `6201e7d144874370b4da432a529014d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:27.992216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:28.047157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1189/2385] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:28.473527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:29.470121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:30.271897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:30.511609 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:30.631735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:30.743568 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:30.832056 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.140345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.162855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.331831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.392116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.432159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.583780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.724228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:31.769898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.226228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.306276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.420710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.446257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.615547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.658165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.825977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:32.888828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.006342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.105358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.131952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.323200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.483331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.508758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.649067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.671647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.845898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:33.909042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:34.106357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:34.255039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:34.426945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:34.520526 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:34.617462 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:34.674588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:39.556583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a53ca-891a-42c4-9f47-6846edd15bc8 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:39.557080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a53ca-891a-42c4-9f47-6846edd15bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:39.557164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a53ca-891a-42c4-9f47-6846edd15bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:39.578328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-bd1a53ca-891a-42c4-9f47-6846edd15bc8 None None] MFA Rules not processed for user `14bd04e951254ec5bc29999bceeea214`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:39.579537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bd1a53ca-891a-42c4-9f47-6846edd15bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:39.642494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1197/2386] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:44.817888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=29 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:46.039965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:46.143401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b9852-9ffd-4af2-b3f0-786db6910f40 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:46.149263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:46.149895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:46.150172 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:46.150442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] PATH_INFO: `/v3/users/67877d8cb07741de8a288a6f097de783` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:46.151697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=67877d8cb07741de8a288a6f097de783)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:46.159780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:46.488948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67877d8cb07741de8a288a6f097de783'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:46.496685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67877d8cb07741de8a288a6f097de783'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:46.591290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67877d8cb07741de8a288a6f097de783'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:46.592022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67877d8cb07741de8a288a6f097de783'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:46.607740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67877d8cb07741de8a288a6f097de783'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:46.615117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1190/2387] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:46 2022] DELETE /identity/v3/users/67877d8cb07741de8a288a6f097de783 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:46.619849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:46.623498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:46.623498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:46.641503 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:46.642857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:46.721619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:46.727290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1198/2388] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:46.738725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:46.740748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:46.811401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a225b-c35c-4a5b-a9ae-f5c182502d5b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:46.815160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:46.816484 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:46.816866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:46.817327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:46.833636 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:46.835167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:46.848174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1191/2389] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:47.620714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:47.622257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:47.635577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94e7d2-1b17-43ed-9f5b-67347195fb69 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:47.648225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:47.649002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:47.649381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:47.649542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] PATH_INFO: `/v3/projects/f74df4eea28c42f885b7484cf327fbb9` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:47.650751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74df4eea28c42f885b7484cf327fbb9)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:47.653020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:47.869440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74df4eea28c42f885b7484cf327fbb9', 'request_id': 'req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.002940 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74df4eea28c42f885b7484cf327fbb9', 'request_id': 'req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.009801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74df4eea28c42f885b7484cf327fbb9', 'request_id': 'req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.021639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] The token cache is being invalidate because project f74df4eea28c42f885b7484cf327fbb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:48.022010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.022669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.024180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1199/2390] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:47 2022] DELETE /identity/v3/projects/f74df4eea28c42f885b7484cf327fbb9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:48.028133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:48.038668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7612a-fe1b-44e3-bbe8-6c0964a6600b service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:48.056926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:48.094694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:48.095321 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:48.095623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:48.095882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] PATH_INFO: `/v3/users/5898a1397345433c92f4255caf464136` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:48.097590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5898a1397345433c92f4255caf464136)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:48.099583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:48.527830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5898a1397345433c92f4255caf464136'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.535333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5898a1397345433c92f4255caf464136'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.640333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5898a1397345433c92f4255caf464136'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.646169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5898a1397345433c92f4255caf464136'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.656287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5898a1397345433c92f4255caf464136'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:48.667120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1192/2391] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:48 2022] DELETE /identity/v3/users/5898a1397345433c92f4255caf464136 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:49.557490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:49.559621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:49.574166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ac07-5f16-4e6f-8b22-bdbb5d94e828 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:49.688332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:49.689054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:49.689341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:49.689694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] PATH_INFO: `/v3/projects/96409331e9ca437aaf71e4006f3b0242` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:49.691037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96409331e9ca437aaf71e4006f3b0242)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:49.694470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:49.900345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96409331e9ca437aaf71e4006f3b0242', 'request_id': 'req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:49.901563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:49.903086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:49.917253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad15827-067a-4642-9371-a5c43e11716a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:50.032591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:50.033237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:50.034016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:50.034529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] PATH_INFO: `/v3/users/5434fbd6018346bf8ce0fd150b490dd0` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:50.036075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5434fbd6018346bf8ce0fd150b490dd0)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:50.038307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:50.044902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96409331e9ca437aaf71e4006f3b0242', 'request_id': 'req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.054473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96409331e9ca437aaf71e4006f3b0242', 'request_id': 'req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.071837 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] The token cache is being invalidate because project 96409331e9ca437aaf71e4006f3b0242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:50.072186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.073434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.075116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1200/2392] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:49 2022] DELETE /identity/v3/projects/96409331e9ca437aaf71e4006f3b0242 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:50.084193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:50.084193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:50.093254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:50.100155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc37f17-ce72-4925-ba9f-eb0f81e5ed58 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:50.105351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:50.105930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:50.106208 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:50.106479 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] PATH_INFO: `/v3/users/7b3c8b45e7f140919bf324686e34a562` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:50.107783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3c8b45e7f140919bf324686e34a562)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:50.110004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:50.367124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:50.370098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5434fbd6018346bf8ce0fd150b490dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.379433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5434fbd6018346bf8ce0fd150b490dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.454268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5434fbd6018346bf8ce0fd150b490dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.455175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5434fbd6018346bf8ce0fd150b490dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.470909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3c8b45e7f140919bf324686e34a562'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.478694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d28729d2-1423-44b9-b603-2be926531e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5434fbd6018346bf8ce0fd150b490dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.480284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3c8b45e7f140919bf324686e34a562'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.481185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3c8b45e7f140919bf324686e34a562'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.482141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3c8b45e7f140919bf324686e34a562'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.489439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1193/2393] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:49 2022] DELETE /identity/v3/users/5434fbd6018346bf8ce0fd150b490dd0 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:50.494072 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:50.494555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:50.495165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:50.517227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:50.518452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:50.564354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3c8b45e7f140919bf324686e34a562'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:50.595579 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1201/2394] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:50 2022] DELETE /identity/v3/users/7b3c8b45e7f140919bf324686e34a562 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:50.599874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1194/2395] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:50.623104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:50.624632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:50.705373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6a6fb-b737-495f-9701-36a5849add60 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:50.710011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:50.711163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:50.711460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:50.711781 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:50.782836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:50.788991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:50.797083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1202/2396] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:51.439329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:51.442364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:51.455075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76048f-f960-4775-8a5b-00f5e4b09e98 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:51.460730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:51.461735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:51.462013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:51.462452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] PATH_INFO: `/v3/users/50dfe6052ebe493dba087b81d34bc380` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:51.463814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=50dfe6052ebe493dba087b81d34bc380)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:51.465980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:51.831553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:51.834349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:51.841517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50dfe6052ebe493dba087b81d34bc380'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:51.855030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50dfe6052ebe493dba087b81d34bc380'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:51.878093 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d97bf5-5a17-4ed3-a814-defdbfd469b8 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:51.883767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:51.885446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:51.885881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:51.886309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] PATH_INFO: `/v3/projects/55b2fb37bb5a44fe8df7d407c7da7983` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:51.887765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b2fb37bb5a44fe8df7d407c7da7983)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:51.890437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:51.937159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:52.014306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50dfe6052ebe493dba087b81d34bc380'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.015008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50dfe6052ebe493dba087b81d34bc380'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.033009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b2d9c246-0cf1-4024-aa2d-b0a7fb6aa5dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50dfe6052ebe493dba087b81d34bc380'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.044070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1195/2397] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:51 2022] DELETE /identity/v3/users/50dfe6052ebe493dba087b81d34bc380 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:52.047739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.047739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.047739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.067947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:52.069232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.115007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b2fb37bb5a44fe8df7d407c7da7983', 'request_id': 'req-72d71901-3d4b-4d65-99d4-daa489d545c8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.155199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1196/2398] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:52.155725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:52.158753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.172133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e5235-025d-46eb-8357-82ff02655e55 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:52.247283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:52.247934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.248214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.248645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] PATH_INFO: `/v3/projects/12aa7e10c4b74de18431a60befaa324c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.249814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12aa7e10c4b74de18431a60befaa324c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:52.251895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:52.262855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b2fb37bb5a44fe8df7d407c7da7983', 'request_id': 'req-72d71901-3d4b-4d65-99d4-daa489d545c8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.269790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b2fb37bb5a44fe8df7d407c7da7983', 'request_id': 'req-72d71901-3d4b-4d65-99d4-daa489d545c8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.281303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:52.281759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] The token cache is being invalidate because project 55b2fb37bb5a44fe8df7d407c7da7983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:52.282221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.282892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.289676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1203/2399] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:51 2022] DELETE /identity/v3/projects/55b2fb37bb5a44fe8df7d407c7da7983 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:52.290858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:52.292244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.350118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d71901-3d4b-4d65-99d4-daa489d545c8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:52.353899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:52.354511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.354788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.355059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.356799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.369134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:52.370545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881d243-261d-40ad-9936-58cc33a9c0d6 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:52.380378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1204/2400] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:52.381717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca306060-56a8-42fa-9902-8bdff387c2df None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.382000 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca306060-56a8-42fa-9902-8bdff387c2df None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.382295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca306060-56a8-42fa-9902-8bdff387c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.407583 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12aa7e10c4b74de18431a60befaa324c', 'request_id': 'req-fb8f48a2-317d-4d79-ac32-85c7418a8d47'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.409101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ca306060-56a8-42fa-9902-8bdff387c2df None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:52.411733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ca306060-56a8-42fa-9902-8bdff387c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.479985 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1205/2401] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:52.488945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.489240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.490255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.509564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:52.510788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.517128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12aa7e10c4b74de18431a60befaa324c', 'request_id': 'req-fb8f48a2-317d-4d79-ac32-85c7418a8d47'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.519912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1206/2402] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:52.524135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:52.525292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12aa7e10c4b74de18431a60befaa324c', 'request_id': 'req-fb8f48a2-317d-4d79-ac32-85c7418a8d47'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.533943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308647e7-6eda-4431-b48f-72b26049a698 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:52.538159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] The token cache is being invalidate because project 12aa7e10c4b74de18431a60befaa324c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:52.538666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.539322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.539739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:52.540398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.540681 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.540880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1197/2403] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:52 2022] DELETE /identity/v3/projects/12aa7e10c4b74de18431a60befaa324c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:52.540992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.541754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:52.542071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:52.543885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:52.545594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:52.547142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.560046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f48a2-317d-4d79-ac32-85c7418a8d47 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:52.561201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1207/2404] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:06:52.562099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:52.563913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:52.565409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:52.566122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.566403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.567031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] PATH_INFO: `/v3/users/1546f49729794a68b8960b35477648f3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.568191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1546f49729794a68b8960b35477648f3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:52.570263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:52.577062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77507054-1211-4ae1-887a-25130fe924fb admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:52.583102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:52.583719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.584134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.584417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.585314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:52.587275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:52.727610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1208/2405] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:52 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:52.731410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:52.822714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d223fe2-36f7-4b8c-8e76-cc4b99516acb admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:52.827551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:52.828164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:52.828533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:52.828811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:52.829447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:52.831403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:52.859252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1546f49729794a68b8960b35477648f3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:52.875432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1546f49729794a68b8960b35477648f3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.041299 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1209/2406] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:52 2022] POST /identity/v3/users => generated 289 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:53.046460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.057108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e3dc-7ff6-4995-abd0-6b3c8723f88c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.062242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.062928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.063211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.063482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.064155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.066003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.079879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1210/2407] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:53.080770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.090600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9301aa-5575-4126-930e-ffe1175ac825 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.095251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.095849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.096135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.096438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.097613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.099186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.111727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1546f49729794a68b8960b35477648f3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.111727 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1546f49729794a68b8960b35477648f3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.116180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1211/2408] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:53.116180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.116180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.128511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a17b60-4d16-4f30-83ac-7cd19eebc296 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.138302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1546f49729794a68b8960b35477648f3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.141513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1198/2409] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:06:52 2022] DELETE /identity/v3/users/1546f49729794a68b8960b35477648f3 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:53.142710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.144233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.144528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.156427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.157155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.157473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] PATH_INFO: `/v3/projects/beafb1d075f047a38091a01899dc4ca7` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.159987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beafb1d075f047a38091a01899dc4ca7)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.162167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.226260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6ea70-6ba4-4df3-9022-72e3553ccb3c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.231807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a651f8c-71a6-4956-a64e-c3cf4d52d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.232525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651f8c-71a6-4956-a64e-c3cf4d52d8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.232812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651f8c-71a6-4956-a64e-c3cf4d52d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.235567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651f8c-71a6-4956-a64e-c3cf4d52d8e5 admin admin] PATH_INFO: `/v3/projects/951beb6763fe43c6929211268a75b36b/users/081e4cc164604c6c82124be618a9829b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.263326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a651f8c-71a6-4956-a64e-c3cf4d52d8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951beb6763fe43c6929211268a75b36b, user_id=081e4cc164604c6c82124be618a9829b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.266304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a651f8c-71a6-4956-a64e-c3cf4d52d8e5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.443685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1199/2410] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:53 2022] PUT /identity/v3/projects/951beb6763fe43c6929211268a75b36b/users/081e4cc164604c6c82124be618a9829b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:53.452230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beafb1d075f047a38091a01899dc4ca7', 'request_id': 'req-0b7a1f62-0e4e-4a48-baae-e600539808f0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.453699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.454304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.454805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.487054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] MFA Rules not processed for user `081e4cc164604c6c82124be618a9829b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:53.488422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.531816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beafb1d075f047a38091a01899dc4ca7', 'request_id': 'req-0b7a1f62-0e4e-4a48-baae-e600539808f0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.539362 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beafb1d075f047a38091a01899dc4ca7', 'request_id': 'req-0b7a1f62-0e4e-4a48-baae-e600539808f0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.558370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] The token cache is being invalidate because project beafb1d075f047a38091a01899dc4ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:53.558370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.558370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0b7a1f62-0e4e-4a48-baae-e600539808f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.558370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1212/2411] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:53 2022] DELETE /identity/v3/projects/beafb1d075f047a38091a01899dc4ca7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:53.558370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9423db-9cd2-4328-8f6c-4f41b22d36ec None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.558920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9423db-9cd2-4328-8f6c-4f41b22d36ec None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.558920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9423db-9cd2-4328-8f6c-4f41b22d36ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.562914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1200/2412] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:53.575469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.576874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.579700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-dd9423db-9cd2-4328-8f6c-4f41b22d36ec None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:53.581106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dd9423db-9cd2-4328-8f6c-4f41b22d36ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.632200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587a36a-1d54-4076-9437-c19d416b91dd None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.635860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.636499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.636780 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.637059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.638800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.639311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1213/2413] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:53.647891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.648178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.648486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.650986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.652500 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.662219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1201/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:06:53.668031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:53.669271 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.677623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1214/2415] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:53.682015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.693880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e08a3-3040-49ff-98fc-c6147837e4d9 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.701911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.701911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.701911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.701911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.702312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:53.702312 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.703847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.704652 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.706719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:53.718950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1202/2416] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:06:53.723944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.724663 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c3be3-b79d-4e91-b56b-0780a6c1b011 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.733095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.743676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.743676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.743676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] PATH_INFO: `/v3/projects/063c33023e6b431fbf4b108bc54152bc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.743676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063c33023e6b431fbf4b108bc54152bc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.744437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34ef69-034e-45ba-9c22-81cc51d99e4b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.744737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.744836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.744836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.744836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.744836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.745486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.746180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.877213 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1203/2417] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:53 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:53.882088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:53.890179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063c33023e6b431fbf4b108bc54152bc', 'request_id': 'req-5155a81d-9932-48c7-a1c0-5358446d7742'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.945238 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ea1a-9431-4427-b64e-98f74816b190 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:53.950527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:53.951144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:53.951425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:53.951698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:53.952364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:53.954314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:53.985709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063c33023e6b431fbf4b108bc54152bc', 'request_id': 'req-5155a81d-9932-48c7-a1c0-5358446d7742'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:53.993099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063c33023e6b431fbf4b108bc54152bc', 'request_id': 'req-5155a81d-9932-48c7-a1c0-5358446d7742'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:54.005055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] The token cache is being invalidate because project 063c33023e6b431fbf4b108bc54152bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:06:54.005424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:54.006130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5155a81d-9932-48c7-a1c0-5358446d7742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:06:54.007690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1215/2418] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:06:53 2022] DELETE /identity/v3/projects/063c33023e6b431fbf4b108bc54152bc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:54.012351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae19b9-57a7-4735-927b-3d273b4def9b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.012640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae19b9-57a7-4735-927b-3d273b4def9b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.012917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae19b9-57a7-4735-927b-3d273b4def9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.034514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3aae19b9-57a7-4735-927b-3d273b4def9b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:54.035897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3aae19b9-57a7-4735-927b-3d273b4def9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:54.051692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1216/2419] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:54.053218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.053506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.053834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.070223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1204/2420] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:53 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:54.074243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.076300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:54.080084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:54.081597 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:54.089518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697fe5-b24f-4371-adfb-8e7025e85616 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.092366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1217/2421] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:06:54.096655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.102752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.102752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.102752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.102752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.102752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.103405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.111355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1205/2422] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:54.113521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71cf8e-2fa2-49b5-9857-32ed02f22c66 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.115752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.118994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.119632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.119909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.120180 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.120872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:06:54.121277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.122794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.129618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc54d4-6176-4868-9070-39a3e8bb0766 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.134883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.135492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.135776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.135882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1218/2423] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:06:54.136118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.136946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.138705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.140354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.151377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1206/2424] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:54.151559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747e118-8bcc-42d9-8406-e0e1ac99b7f6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.156643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fbb18-acdc-4e7a-94e7-118f88c957cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.157384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fbb18-acdc-4e7a-94e7-118f88c957cd admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.157766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fbb18-acdc-4e7a-94e7-118f88c957cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.158270 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fbb18-acdc-4e7a-94e7-118f88c957cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.159351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fbb18-acdc-4e7a-94e7-118f88c957cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.161568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fbb18-acdc-4e7a-94e7-118f88c957cd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.164694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.176661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc13ed-7b1c-4a28-a2e2-207ca56dc79b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.182339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.183039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.183328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.183603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] PATH_INFO: `/v3/projects/0a6caf631e5849f6867042a1f6dcf6aa/users/9132e6a676054c93ba33d95d28ded4da/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.200664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6caf631e5849f6867042a1f6dcf6aa, user_id=9132e6a676054c93ba33d95d28ded4da, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.203326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.619050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1207/2425] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:54 2022] PUT /identity/v3/projects/0a6caf631e5849f6867042a1f6dcf6aa/users/9132e6a676054c93ba33d95d28ded4da/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:54.620086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1219/2426] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:54 2022] POST /identity/v3/projects => generated 377 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:54.620683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.622080 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c2157-00c2-4626-911c-e05551f58cef None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.622913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c2157-00c2-4626-911c-e05551f58cef None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.622913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c2157-00c2-4626-911c-e05551f58cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.645829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-431c2157-00c2-4626-911c-e05551f58cef None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:54.659363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-431c2157-00c2-4626-911c-e05551f58cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:54.712471 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c70038-32b8-4b35-8ec3-887ddd40129b admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.728085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1220/2427] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:54.738585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c2157-00c2-4626-911c-e05551f58cef None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:54.740280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-431c2157-00c2-4626-911c-e05551f58cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:54.794853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.795531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.795818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.796093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.796808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.798883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.808812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c2157-00c2-4626-911c-e05551f58cef None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:54.813390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:54.814314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:54.814771 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:54.815044 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:54.826358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:54.827798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:54.837832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1221/2428] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:06:55.038435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1208/2429] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:54 2022] POST /identity/v3/users => generated 289 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:06:55.043029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:55.055719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd04eaf-38b0-432c-b033-626f8d03e52c service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:55.062688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:55.063424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:55.063951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:55.064900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:55.065204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:55.067649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:55.082055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1222/2430] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:55.086178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:55.101884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad572-d254-4ef1-9957-e93f7902d2c3 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:55.115602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe435da3-65f1-43ec-a88c-97ce8743a314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:55.117113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435da3-65f1-43ec-a88c-97ce8743a314 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:55.117113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435da3-65f1-43ec-a88c-97ce8743a314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:55.117446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435da3-65f1-43ec-a88c-97ce8743a314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:55.118300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe435da3-65f1-43ec-a88c-97ce8743a314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:55.121059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe435da3-65f1-43ec-a88c-97ce8743a314 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:55.135075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1209/2431] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:55.173811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:55.187331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77f060-fd3e-485c-a1a7-af8fe2da9012 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:55.193923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:55.195170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:55.195733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:55.196184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] PATH_INFO: `/v3/projects/6b0027c75398414eb371c568527fb663/users/1aa8b82dfa104202b37a9da94feb0208/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:55.233825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0027c75398414eb371c568527fb663, user_id=1aa8b82dfa104202b37a9da94feb0208, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:55.237227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:55.575639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1223/2432] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:55 2022] PUT /identity/v3/projects/6b0027c75398414eb371c568527fb663/users/1aa8b82dfa104202b37a9da94feb0208/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:06:55.588342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:55.588668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:55.589263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:55.628368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] MFA Rules not processed for user `1aa8b82dfa104202b37a9da94feb0208`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:55.630317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:55.719604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1210/2433] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:55.725686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:55.798453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34d17f-a455-4c39-9c44-0a93cc5dd2d4 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:55.803441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:55.804146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:55.804529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:55.804969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:55.805536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:55.807488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:55.995797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1224/2434] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:55 2022] POST /identity/v3/projects => generated 375 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:55.997189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:56.008908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062723a-cc63-4efe-9e0f-614985ce8980 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:56.076330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:56.083401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.083401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.083401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.083401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:56.083401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:56.177336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:56.246179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1211/2435] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:55 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:56.247439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:56.258827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709d573-1f36-4860-a694-0a75d04b9b10 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:56.263851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:56.264510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.264794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.265068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.265726 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:56.267311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:56.278946 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1225/2436] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:56.282735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:56.294289 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207db88-c9fe-4acc-9904-b0253d1b987f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:56.301437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0415697-9609-4401-ac5d-e7a503b43225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:56.302324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0415697-9609-4401-ac5d-e7a503b43225 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.302460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0415697-9609-4401-ac5d-e7a503b43225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.302705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0415697-9609-4401-ac5d-e7a503b43225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.303491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0415697-9609-4401-ac5d-e7a503b43225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:56.305119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0415697-9609-4401-ac5d-e7a503b43225 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:56.320788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1212/2437] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:06:56.322011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:56.333473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592f164-024d-4cd2-854c-0a12a66e0e63 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:56.338335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:56.339103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.339562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.340048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] PATH_INFO: `/v3/projects/610c1fc7ec4248209a2166583df8d01d/users/767081c7641d4b5db072e35bfa78f66d/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.364856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] RBAC: Authorizing `identity:create_grant(project_id=610c1fc7ec4248209a2166583df8d01d, user_id=767081c7641d4b5db072e35bfa78f66d, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:56.367753 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:56.443229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:06:56.485619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1226/2438] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:56 2022] PUT /identity/v3/projects/610c1fc7ec4248209a2166583df8d01d/users/767081c7641d4b5db072e35bfa78f66d/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:56.497445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.497725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.498020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.527275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] MFA Rules not processed for user `767081c7641d4b5db072e35bfa78f66d`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:56.528534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:56.604769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:56.605158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1213/2439] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:56.674780 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4880-e2c8-4c41-8eed-46d15af3419e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:56.680458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:56.681075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.681356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.681632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.682272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:56.684150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:56.844289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1227/2440] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:06:56 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:56.848889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:56.859905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb0c98-d102-452a-aade-057a2f9e6819 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:56.922179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:56.922915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:56.923265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:56.923974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:56.924328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:56.927167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:57.072953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1214/2441] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:06:56 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:06:57.076436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:57.088554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32211ae-2ea4-49c3-9194-bf3782b909c2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:57.094926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:57.095498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:57.095776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:57.096048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:57.097366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:57.098957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:57.115537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1228/2442] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:57.117183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:57.127779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09288e16-f8b6-4d9b-996a-2aa840069316 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:57.134019 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:57.134810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:57.135191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:57.135601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] PATH_INFO: `/v3/projects/5d40312531894ca19427d7e739b6b160/users/cdf30f6fa39844e48209b328be638ef3/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:57.153585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d40312531894ca19427d7e739b6b160, user_id=cdf30f6fa39844e48209b328be638ef3, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:57.156826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:57.328235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1215/2443] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:57 2022] PUT /identity/v3/projects/5d40312531894ca19427d7e739b6b160/users/cdf30f6fa39844e48209b328be638ef3/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:57.332691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:57.342574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a7f52-0535-4fb5-8399-b4dc85616e32 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:57.405129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3644135-f40a-4cca-adcd-2817265b2596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:57.405466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644135-f40a-4cca-adcd-2817265b2596 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:57.405839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644135-f40a-4cca-adcd-2817265b2596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:57.406204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644135-f40a-4cca-adcd-2817265b2596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:57.407088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3644135-f40a-4cca-adcd-2817265b2596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:57.410821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3644135-f40a-4cca-adcd-2817265b2596 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:57.434151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1229/2444] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:06:57.438713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:57.462613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ce476-eb6a-41ba-b334-e72cc8580620 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:57.475677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:57.483681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:57.483681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:57.483681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] PATH_INFO: `/v3/projects/5d40312531894ca19427d7e739b6b160/users/cdf30f6fa39844e48209b328be638ef3/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:57.483681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d40312531894ca19427d7e739b6b160, user_id=cdf30f6fa39844e48209b328be638ef3, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:57.483681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:57.506082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1216/2445] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:06:57 2022] PUT /identity/v3/projects/5d40312531894ca19427d7e739b6b160/users/cdf30f6fa39844e48209b328be638ef3/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:06:57.509254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119a83b-1577-4241-b0e4-f9c7dba2aaad None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:57.509814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119a83b-1577-4241-b0e4-f9c7dba2aaad None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:57.510351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119a83b-1577-4241-b0e4-f9c7dba2aaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:57.686253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-9119a83b-1577-4241-b0e4-f9c7dba2aaad None None] MFA Rules not processed for user `cdf30f6fa39844e48209b328be638ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:06:57.687855 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9119a83b-1577-4241-b0e4-f9c7dba2aaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:57.779336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1230/2446] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:06:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:06:57.792563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:06:57.795564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:06:57.900862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543de72-2451-446b-8bc7-871ef46f4f76 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:06:57.904139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:06:57.904814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:06:57.911880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:06:57.911880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:06:58.019022 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:06:58.021447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:06:58.043798 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1217/2447] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:06:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:06:59.652047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:00.071973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:01.415887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:01.555404 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:01.649947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:01.650455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:01.650733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:01.698648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] MFA Rules not processed for user `9132e6a676054c93ba33d95d28ded4da`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:01.699895 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:01.772071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1231/2448] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:01.775806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:01.843015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbbde6-7dcf-4724-a473-8701db70bbc5 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:01.848482 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:01.849097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:01.849377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:01.849645 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:01.850296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:01.852207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:02.001819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1218/2449] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:07:01 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:07:02.002027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:02.002027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f6db3-4c9c-495b-9631-b494f95daac0 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:02.070440 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:02.071169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:02.071451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:02.071826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:02.072566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:02.074617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:02.205673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1232/2450] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:07:01 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:02.209761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:02.220102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1098c5-f8ec-4a23-853e-9ee9ba27c2e3 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:02.224762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:02.229325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:02.229325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:02.229325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:02.229325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:02.229325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:02.239802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1219/2451] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:02.243858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:02.255053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbecfd-9e0c-42e8-a4ff-fe0d20a7cba9 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:02.259748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:02.263749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:02.263749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:02.263749 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] PATH_INFO: `/v3/projects/523583963f30429a9a36115c85777964/users/da283172c12f4a81b4c5c87fd138eec8/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:02.275850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523583963f30429a9a36115c85777964, user_id=da283172c12f4a81b4c5c87fd138eec8, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:02.278585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:02.363114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1233/2452] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:02 2022] PUT /identity/v3/projects/523583963f30429a9a36115c85777964/users/da283172c12f4a81b4c5c87fd138eec8/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:02.367269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:02.377675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd35c2-4d75-433e-b234-fd61933e96f8 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:02.445374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:02.446084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:02.446360 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:02.446633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:02.447322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:02.449000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:02.459761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1220/2453] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:02.463822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:02.474611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a13a4-f66d-415c-a76d-181c8f837247 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:02.479314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:02.480032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:02.480333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:02.480640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] PATH_INFO: `/v3/projects/523583963f30429a9a36115c85777964/users/da283172c12f4a81b4c5c87fd138eec8/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:02.483727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523583963f30429a9a36115c85777964, user_id=da283172c12f4a81b4c5c87fd138eec8, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:02.485373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:02.496959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1234/2454] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:02 2022] PUT /identity/v3/projects/523583963f30429a9a36115c85777964/users/da283172c12f4a81b4c5c87fd138eec8/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:02.602799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:02.717734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:03.287906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:03.289818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:03.370514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d2327-2d47-458e-8dbf-e2c2d38f0945 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:03.431829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:03.432475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:03.432757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:03.433033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:03.434787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:03.446953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:03.448575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:03.458965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1221/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:07:04.288908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:04.531707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:04.546048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:04.548449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:04.656798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdffdf5-5a70-406a-98d5-d18581d63622 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:04.665242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac81e4-b91f-469e-9f7e-8ce4fac3dd84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:04.666463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac81e4-b91f-469e-9f7e-8ce4fac3dd84 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:04.667572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac81e4-b91f-469e-9f7e-8ce4fac3dd84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:04.668881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac81e4-b91f-469e-9f7e-8ce4fac3dd84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:04.814491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac81e4-b91f-469e-9f7e-8ce4fac3dd84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:04.817814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac81e4-b91f-469e-9f7e-8ce4fac3dd84 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:04.840490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1235/2456] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:06.448669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:06.525988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:08.142779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:08.152297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567af6-503f-48b6-ac05-a16e3dafca2b service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:08.156437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:08.156641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:08.156920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:08.157192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:08.260070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:08.261673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:08.280454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1222/2457] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:09.339893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf874c4-f355-4f5f-b41a-1326a237afa5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:09.340456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf874c4-f355-4f5f-b41a-1326a237afa5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:09.343905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf874c4-f355-4f5f-b41a-1326a237afa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:09.372365 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fdf874c4-f355-4f5f-b41a-1326a237afa5 None None] MFA Rules not processed for user `da283172c12f4a81b4c5c87fd138eec8`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:09.372549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fdf874c4-f355-4f5f-b41a-1326a237afa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:09.465451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1236/2458] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:09.482510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:09.484874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:09.520607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:09.569516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43775f-b318-44d7-8b83-39aa7ebdb9d6 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:09.575548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:09.577347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:09.577347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:09.577347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:09.649159 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:09.654606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:09.670456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1223/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:13.852089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:13.852573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:13.852675 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:13.874312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] MFA Rules not processed for user `9132e6a676054c93ba33d95d28ded4da`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:13.875815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:13.887213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1237/2460] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:07:16.925893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:16.995675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d8144-c3eb-4090-b901-49472bbdf6f4 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:17.001557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:17.002211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:17.002488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:17.002939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] PATH_INFO: `/v3/users/1aa8b82dfa104202b37a9da94feb0208` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:17.004090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa8b82dfa104202b37a9da94feb0208)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:17.006179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:17.186511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:17.244719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cad36-b5a8-4b64-bad6-8cf6d5aec0fe None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:17.248366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:17.252535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:17.252535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:17.252535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:17.252535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:17.335713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:17.337266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa1ac1-1a98-4c0b-b58a-376a1806ccde service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:17.347731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1238/2461] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:07:17.420692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa8b82dfa104202b37a9da94feb0208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:17.430349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa8b82dfa104202b37a9da94feb0208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:17.605030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa8b82dfa104202b37a9da94feb0208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:17.605648 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa8b82dfa104202b37a9da94feb0208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:17.608029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:17.618624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa8b82dfa104202b37a9da94feb0208'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:17.624690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1224/2462] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:16 2022] DELETE /identity/v3/users/1aa8b82dfa104202b37a9da94feb0208 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:17.629601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:17.629905 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:17.630174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:17.643412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:17.649371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:17.650557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:17.709696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1239/2463] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:17.719893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:17.724598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:17.803696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60cd4-c60b-476c-8dbe-e880cd5d46dc admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:17.807618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:17.808244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:17.808552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:17.808826 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:17.820192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:17.821765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:17.833114 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1225/2464] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:07:18.563676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:18.565799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:18.577939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11fc6a-35b8-4355-8e5e-123803e4f6e4 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:18.583617 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:18.584480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:18.584975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:18.585486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] PATH_INFO: `/v3/projects/6b0027c75398414eb371c568527fb663` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:18.586923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0027c75398414eb371c568527fb663)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:18.588928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:18.752592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0027c75398414eb371c568527fb663', 'request_id': 'req-5da38b73-779e-45bd-b648-6b2443bead90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:18.896628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0027c75398414eb371c568527fb663', 'request_id': 'req-5da38b73-779e-45bd-b648-6b2443bead90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:18.902931 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0027c75398414eb371c568527fb663', 'request_id': 'req-5da38b73-779e-45bd-b648-6b2443bead90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:18.914350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] The token cache is being invalidate because project 6b0027c75398414eb371c568527fb663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:18.914638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:18.915258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:18.916762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1240/2465] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:18 2022] DELETE /identity/v3/projects/6b0027c75398414eb371c568527fb663 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:18.920703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:18.931444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb1fe8-a0ff-478a-817a-a58c8ffc8939 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:18.996917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:18.997654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:18.998038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:18.998322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] PATH_INFO: `/v3/users/767081c7641d4b5db072e35bfa78f66d` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:18.999468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] RBAC: Authorizing `identity:delete_user(user_id=767081c7641d4b5db072e35bfa78f66d)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:19.001523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:19.340421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767081c7641d4b5db072e35bfa78f66d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:19.348191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767081c7641d4b5db072e35bfa78f66d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:19.534253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767081c7641d4b5db072e35bfa78f66d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:19.534996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767081c7641d4b5db072e35bfa78f66d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:19.548801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767081c7641d4b5db072e35bfa78f66d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:19.555383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1226/2466] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:18 2022] DELETE /identity/v3/users/767081c7641d4b5db072e35bfa78f66d => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:20.310687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:20.312831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:20.332833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da38b73-779e-45bd-b648-6b2443bead90 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:20.409245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:20.409901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:20.410179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:20.410458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] PATH_INFO: `/v3/projects/610c1fc7ec4248209a2166583df8d01d` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:20.411639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=610c1fc7ec4248209a2166583df8d01d)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:20.414526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:20.660869 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:20.673226 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '610c1fc7ec4248209a2166583df8d01d', 'request_id': 'req-cb0c2481-564a-45c3-a4b2-32c2e22444ba'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:20.863526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '610c1fc7ec4248209a2166583df8d01d', 'request_id': 'req-cb0c2481-564a-45c3-a4b2-32c2e22444ba'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:20.875850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '610c1fc7ec4248209a2166583df8d01d', 'request_id': 'req-cb0c2481-564a-45c3-a4b2-32c2e22444ba'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:20.889132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] The token cache is being invalidate because project 610c1fc7ec4248209a2166583df8d01d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:20.889582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:20.890439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:20.892030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1241/2467] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:20 2022] DELETE /identity/v3/projects/610c1fc7ec4248209a2166583df8d01d => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:20.896194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:20.897680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:20.906902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:20.911749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ef478-55dc-4f51-8144-8b3956dd873c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:20.974907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:20.975700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:20.976087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:20.976414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] PATH_INFO: `/v3/users/cdf30f6fa39844e48209b328be638ef3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:20.977557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf30f6fa39844e48209b328be638ef3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:20.979527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:21.191898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf30f6fa39844e48209b328be638ef3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:21.199554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf30f6fa39844e48209b328be638ef3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:21.297715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf30f6fa39844e48209b328be638ef3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:21.299300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf30f6fa39844e48209b328be638ef3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:21.313698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e175586c-863a-4666-963a-aa7dd501ca71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf30f6fa39844e48209b328be638ef3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:21.320904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1227/2468] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:20 2022] DELETE /identity/v3/users/cdf30f6fa39844e48209b328be638ef3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:21.803538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:21.805517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:21.821482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2481-564a-45c3-a4b2-32c2e22444ba admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:21.883240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:21.883888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:21.884171 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:21.884488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] PATH_INFO: `/v3/projects/5d40312531894ca19427d7e739b6b160` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:21.885607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d40312531894ca19427d7e739b6b160)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:21.887608 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.037278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d40312531894ca19427d7e739b6b160', 'request_id': 'req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:22.167811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d40312531894ca19427d7e739b6b160', 'request_id': 'req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:22.175873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d40312531894ca19427d7e739b6b160', 'request_id': 'req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:22.187302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] The token cache is being invalidate because project 5d40312531894ca19427d7e739b6b160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:22.187593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:22.188255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c8be6d54-8da0-4998-9e3e-4f88258ad1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:22.189770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1242/2469] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:21 2022] DELETE /identity/v3/projects/5d40312531894ca19427d7e739b6b160 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:22.194671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.195148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.195260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.214915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:22.216128 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:22.280546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1228/2470] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:22.290394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.290612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.290891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.312680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:22.313970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:22.322430 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1243/2471] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:22.326757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:22.336904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a1d4-d0a5-41bf-8875-ac5ee03b0611 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:22.341699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:22.342305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.342587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.342858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.343551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:07:22.343856 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:22.350943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.362833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1229/2472] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:07:22.367310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:22.376435 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050209-e769-4fe5-885c-8288cfa7a791 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:22.394040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:22.394040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.394040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.394040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.394477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:22.394477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.507600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1244/2473] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:07:22 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:22.511318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:22.522071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471c47a-6357-433c-9d23-2f7bca37ded2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:22.529877 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:22.581817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:22.582420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.582696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.582962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.583592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:22.585522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.719558 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1230/2474] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:07:22 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:22.724456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:22.734515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58647712-61b6-43b0-b891-87056736cd44 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:22.739195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:22.739793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.740062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.740377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.741025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:22.742604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.755402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1245/2475] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:22.759400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:22.770042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44de3d7-128f-4cb0-b0e5-a321198572c2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:22.774709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117b009-ead0-4b8a-8299-af24fb049462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:22.775301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b009-ead0-4b8a-8299-af24fb049462 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.775580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b009-ead0-4b8a-8299-af24fb049462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.775853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b009-ead0-4b8a-8299-af24fb049462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.776527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117b009-ead0-4b8a-8299-af24fb049462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:22.778120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117b009-ead0-4b8a-8299-af24fb049462 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.789707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1231/2476] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:22.793810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:22.804911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee32a3d-72f7-4fd2-b9c5-515e9ae51e76 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:22.809459 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:22.810138 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.810413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.810688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] PATH_INFO: `/v3/projects/6338e85281e5474aab0f5fe0158e82ca/users/9f5f039248434c8d992de5b4ed271211/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:22.819850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:22.827563 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6338e85281e5474aab0f5fe0158e82ca, user_id=9f5f039248434c8d992de5b4ed271211, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:22.830765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:22.976634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1246/2477] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:22 2022] PUT /identity/v3/projects/6338e85281e5474aab0f5fe0158e82ca/users/9f5f039248434c8d992de5b4ed271211/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:22.987278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:22.987704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:22.987956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.015979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] MFA Rules not processed for user `9f5f039248434c8d992de5b4ed271211`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:23.017846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:23.082455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1232/2478] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:23.087205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:23.153685 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094d29-9919-4290-8d48-f39a45efe3d8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:23.164185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:23.164823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:23.165167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:23.165480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.166191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:23.168209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:23.328018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1247/2479] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:07:23 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:23.333059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:23.344164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3691d-6559-4b5f-be9f-61899d180a34 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:23.409162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:23.409162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:23.409162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:23.409162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.409510 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:23.409958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:23.587060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1233/2480] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:07:23 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:07:23.591112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:23.602050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddef7f-6555-4139-95b3-b38a2737fc64 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:23.607911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:23.607911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:23.607911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:23.608466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.608658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:23.610257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:23.621206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1248/2481] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:23.628241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:23.638391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f4bb1-79fb-446d-a034-9947b2ba1be1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:23.643789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937569d4-0d0b-4d15-b7e7-23bdc5e674ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:23.644417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937569d4-0d0b-4d15-b7e7-23bdc5e674ab admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:23.644698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937569d4-0d0b-4d15-b7e7-23bdc5e674ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:23.644974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937569d4-0d0b-4d15-b7e7-23bdc5e674ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.645621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937569d4-0d0b-4d15-b7e7-23bdc5e674ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:23.647189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937569d4-0d0b-4d15-b7e7-23bdc5e674ab admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:23.649336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:23.657925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1234/2482] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:23.662421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:23.672656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727457a8-5eaa-40ea-951e-cfb1c331216d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:23.677312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:23.677959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:23.678240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:23.678514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] PATH_INFO: `/v3/projects/0c2889ebfeff4ef98f8bec40b79698c8/users/312341fed6fc4dff9ff9c3f5bbffa6df/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.693460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2889ebfeff4ef98f8bec40b79698c8, user_id=312341fed6fc4dff9ff9c3f5bbffa6df, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:23.696121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:23.847132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1249/2483] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:23 2022] PUT /identity/v3/projects/0c2889ebfeff4ef98f8bec40b79698c8/users/312341fed6fc4dff9ff9c3f5bbffa6df/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:23.850397 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:23.850713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:23.850992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:23.879750 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] MFA Rules not processed for user `312341fed6fc4dff9ff9c3f5bbffa6df`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:23.881034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:23.950077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1235/2484] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:23.955694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.016949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613847-007e-480c-9294-d5296e3748dd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.021629 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.022228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.022510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.022783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.023415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.025318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:24.155075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1250/2485] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:07:23 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:07:24.159591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.170091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e028e-d0ce-4927-9d87-1e787d53368b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.234447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.235055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.235337 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.235611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.236844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.238770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:24.349794 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1236/2486] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:07:24 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:24.355119 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.366346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a2c10-d5ed-4ee8-bbf3-9b00f2eaac77 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.370932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.371532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.371812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.372078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.372759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.374315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:24.384977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1251/2487] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:24.389168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.399422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941a01b-6227-414f-b89f-9c70959e4a4c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.403895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.404565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.404846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.405169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] PATH_INFO: `/v3/projects/64b1715258cf41f6aa0dadfc9ff29164/users/0c9c02792bba494dac9fc496b889f9b0/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.420623 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b1715258cf41f6aa0dadfc9ff29164, user_id=0c9c02792bba494dac9fc496b889f9b0, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.423247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:24.545647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1237/2488] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:24 2022] PUT /identity/v3/projects/64b1715258cf41f6aa0dadfc9ff29164/users/0c9c02792bba494dac9fc496b889f9b0/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:24.550291 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.560286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933977b1-cc0e-4f16-acc3-2f44c84c6913 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.568281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:24.615665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841e2c-ff63-4e40-b7c0-cfb2edaea2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.616265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841e2c-ff63-4e40-b7c0-cfb2edaea2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.616580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841e2c-ff63-4e40-b7c0-cfb2edaea2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.616853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841e2c-ff63-4e40-b7c0-cfb2edaea2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.617493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65841e2c-ff63-4e40-b7c0-cfb2edaea2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.619051 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65841e2c-ff63-4e40-b7c0-cfb2edaea2a9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:24.628558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1252/2489] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:24.633024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.652186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efed53-cc8c-4dd3-b2a3-0e480cd55c23 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.657046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.658353 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.658768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.658968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] PATH_INFO: `/v3/projects/64b1715258cf41f6aa0dadfc9ff29164/users/0c9c02792bba494dac9fc496b889f9b0/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.661090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b1715258cf41f6aa0dadfc9ff29164, user_id=0c9c02792bba494dac9fc496b889f9b0, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.663928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:24.676505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1238/2490] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:24 2022] PUT /identity/v3/projects/64b1715258cf41f6aa0dadfc9ff29164/users/0c9c02792bba494dac9fc496b889f9b0/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:24.745063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.745350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.745623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.773936 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] MFA Rules not processed for user `0c9c02792bba494dac9fc496b889f9b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:24.775142 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:24.846684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1253/2491] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:07:24.862873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:24.862873 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:24.912620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:24.926190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e35c1f-3255-41e5-b117-8c561840f032 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:24.929819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:24.930423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:24.930701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:24.930974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:24.989995 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:24.991416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:25.000028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1239/2492] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:31.457321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:31.458742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:31.470708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc62f7-c626-4f4d-add2-e4001e3cfb17 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:31.474574 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:31.475237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:31.475561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:31.475836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:31.537958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:31.539397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:31.548976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1254/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:33.731302 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:33.741045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af29d72-e069-48b8-a0f1-d65c22d0f535 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:33.744657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d08f23-ff09-4cd6-a90b-977dc072eb36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:33.745334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d08f23-ff09-4cd6-a90b-977dc072eb36 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:33.745624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d08f23-ff09-4cd6-a90b-977dc072eb36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:33.745940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d08f23-ff09-4cd6-a90b-977dc072eb36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:33.755800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d08f23-ff09-4cd6-a90b-977dc072eb36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:33.757258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d08f23-ff09-4cd6-a90b-977dc072eb36 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:33.766701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1240/2494] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:37.252703 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:37.254193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:37.288912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:37.319851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb6e15-6810-443a-802a-9d966bd775e9 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:37.324588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:37.325164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:37.325441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:37.325710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] PATH_INFO: `/v3/users/081e4cc164604c6c82124be618a9829b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:37.326821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081e4cc164604c6c82124be618a9829b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:37.328848 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:37.673018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081e4cc164604c6c82124be618a9829b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:37.680011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081e4cc164604c6c82124be618a9829b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:37.680723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081e4cc164604c6c82124be618a9829b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:37.681341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081e4cc164604c6c82124be618a9829b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:37.781718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081e4cc164604c6c82124be618a9829b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:37.800700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1255/2495] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:37 2022] DELETE /identity/v3/users/081e4cc164604c6c82124be618a9829b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:37.805461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:37.805961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:37.806067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:37.826201 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:37.827357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:37.887669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1241/2496] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:37.898040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:37.902060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:37.968467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33493fe-a078-4f4b-b34f-8452a213a104 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:37.972897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aceb3a-9128-49ad-a017-8c1a62643b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:37.973498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aceb3a-9128-49ad-a017-8c1a62643b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:37.973772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aceb3a-9128-49ad-a017-8c1a62643b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:37.974045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aceb3a-9128-49ad-a017-8c1a62643b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:37.982754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aceb3a-9128-49ad-a017-8c1a62643b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:37.984155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aceb3a-9128-49ad-a017-8c1a62643b26 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:37.994118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1256/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:07:38.503895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:38.505394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:38.518056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e95307-2ebd-4f82-a1ac-5fc32cc83280 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:38.522662 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:38.523293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:38.523570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:38.523838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] PATH_INFO: `/v3/projects/951beb6763fe43c6929211268a75b36b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:38.524981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=951beb6763fe43c6929211268a75b36b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:38.526905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:38.705688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951beb6763fe43c6929211268a75b36b', 'request_id': 'req-8e55f570-3137-41a3-91ec-2789f65d9c34'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:38.895880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951beb6763fe43c6929211268a75b36b', 'request_id': 'req-8e55f570-3137-41a3-91ec-2789f65d9c34'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:38.902101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951beb6763fe43c6929211268a75b36b', 'request_id': 'req-8e55f570-3137-41a3-91ec-2789f65d9c34'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:38.913611 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] The token cache is being invalidate because project 951beb6763fe43c6929211268a75b36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:38.913899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:38.914522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8e55f570-3137-41a3-91ec-2789f65d9c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:38.915997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1242/2498] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:38 2022] DELETE /identity/v3/projects/951beb6763fe43c6929211268a75b36b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:38.920921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:38.921357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:38.921498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:38.942277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:38.943510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:39.001899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:39.015555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1257/2499] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:39.023203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.023489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.023817 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.032335 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:39.044063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:39.045300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:39.054371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1243/2500] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:39.058676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.069338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3440e0-81ee-43f2-a452-2fcf3776ec66 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.073947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.074555 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.074833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.075105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.075754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:07:39.076054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.077643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.088836 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1258/2501] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:07:39.093376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.102762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa22d9-9fa5-4947-b10a-8a5f7ab229f3 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.107445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.108042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.108376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.108650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.109283 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.111709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.261460 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1244/2502] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:07:39 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:07:39.263999 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.274522 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e771c-2c02-4423-8276-766ae31adb3c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.330554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.331264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.331542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.331808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.332465 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.334370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.473757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1259/2503] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:07:39 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:39.478083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.489227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3334eb-d2d1-418d-b3eb-1cf9a71ce533 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.493753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.494370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.494618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.494887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.495533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.497214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.507524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1245/2504] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:07:39.511661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.521852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ab272-82ca-450f-9805-bced60194fda admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.526648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbf3ac-7443-402d-85ff-adcfbc4ddf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.527238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbf3ac-7443-402d-85ff-adcfbc4ddf17 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.527516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbf3ac-7443-402d-85ff-adcfbc4ddf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.527785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbf3ac-7443-402d-85ff-adcfbc4ddf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.528733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbf3ac-7443-402d-85ff-adcfbc4ddf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.530307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbf3ac-7443-402d-85ff-adcfbc4ddf17 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.542073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1260/2505] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:39.546598 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.557637 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d6478-2cd4-447f-b052-92a9703033c1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.562153 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.562793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.563113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.563503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] PATH_INFO: `/v3/projects/bc76188669cb4e6a9549c64de29472ca/users/8df40cd433e240f9b6d3f24b971ffdf8/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.578007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc76188669cb4e6a9549c64de29472ca, user_id=8df40cd433e240f9b6d3f24b971ffdf8, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.580983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.776221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1246/2506] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:39 2022] PUT /identity/v3/projects/bc76188669cb4e6a9549c64de29472ca/users/8df40cd433e240f9b6d3f24b971ffdf8/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:39.788345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.788643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.788917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.822172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] MFA Rules not processed for user `8df40cd433e240f9b6d3f24b971ffdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:39.823439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:39.889301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1261/2507] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:39.901054 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:39.902768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:39.963194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f83ce-88e5-4e75-b6e4-a055fb1925b6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:39.966977 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89526eb2-339c-4d5a-a759-88bf55b19e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:39.967584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89526eb2-339c-4d5a-a759-88bf55b19e61 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:39.967865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89526eb2-339c-4d5a-a759-88bf55b19e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:39.968138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89526eb2-339c-4d5a-a759-88bf55b19e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:39.977305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89526eb2-339c-4d5a-a759-88bf55b19e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:39.978743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89526eb2-339c-4d5a-a759-88bf55b19e61 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:39.989690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1247/2508] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:07:40.136971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:40.138407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:40.187206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:40.203073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c17f7-f68c-478a-8d1f-a6af86719c80 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:40.208218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:40.208878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:40.209161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:40.209438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] PATH_INFO: `/v3/users/9f5f039248434c8d992de5b4ed271211` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:40.210560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5f039248434c8d992de5b4ed271211)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:40.212485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:40.493359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5f039248434c8d992de5b4ed271211'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:40.500015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5f039248434c8d992de5b4ed271211'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:40.500776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5f039248434c8d992de5b4ed271211'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:40.501966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5f039248434c8d992de5b4ed271211'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:40.617003 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5f039248434c8d992de5b4ed271211'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:40.645472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1262/2509] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:40 2022] DELETE /identity/v3/users/9f5f039248434c8d992de5b4ed271211 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:40.650890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:40.651314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:40.651728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:40.678076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:40.679332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:40.742028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1248/2510] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:40.752423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:40.753833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:40.809341 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410cc69-95b8-439d-9d70-02f35078ff43 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:40.813123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:40.814405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:40.814747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:40.815024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:40.823852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:40.825285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:40.834783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1263/2511] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:41.403090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:41.404711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:41.419501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee71d-fc9f-4cc8-8460-d6483225e996 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:41.424187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:41.425408 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:41.425851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:41.426134 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] PATH_INFO: `/v3/projects/6338e85281e5474aab0f5fe0158e82ca` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:41.427271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6338e85281e5474aab0f5fe0158e82ca)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:41.429386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:41.570781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:41.577532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6338e85281e5474aab0f5fe0158e82ca', 'request_id': 'req-e8ec177a-7c98-464f-b95a-bea340c3ecc1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:41.705511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6338e85281e5474aab0f5fe0158e82ca', 'request_id': 'req-e8ec177a-7c98-464f-b95a-bea340c3ecc1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:41.711414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6338e85281e5474aab0f5fe0158e82ca', 'request_id': 'req-e8ec177a-7c98-464f-b95a-bea340c3ecc1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:41.722300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] The token cache is being invalidate because project 6338e85281e5474aab0f5fe0158e82ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:41.722586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:41.723210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:41.724728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1249/2512] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:41 2022] DELETE /identity/v3/projects/6338e85281e5474aab0f5fe0158e82ca => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:41.729132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:41.738826 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf43d4-5afe-4d65-b1d9-71407181416a service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:41.796357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:41.798176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:41.798471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:41.798741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] PATH_INFO: `/v3/users/312341fed6fc4dff9ff9c3f5bbffa6df` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:41.799882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=312341fed6fc4dff9ff9c3f5bbffa6df)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:41.801893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:41.814404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:42.357588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312341fed6fc4dff9ff9c3f5bbffa6df'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:42.364568 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312341fed6fc4dff9ff9c3f5bbffa6df'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:42.365255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312341fed6fc4dff9ff9c3f5bbffa6df'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:42.365854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312341fed6fc4dff9ff9c3f5bbffa6df'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:42.601434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312341fed6fc4dff9ff9c3f5bbffa6df'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:42.618786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1264/2513] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:41 2022] DELETE /identity/v3/users/312341fed6fc4dff9ff9c3f5bbffa6df => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:43.266012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:43.267754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:43.280940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec177a-7c98-464f-b95a-bea340c3ecc1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:43.339729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:43.340385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:43.340664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:43.340931 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] PATH_INFO: `/v3/projects/0c2889ebfeff4ef98f8bec40b79698c8` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:43.350838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2889ebfeff4ef98f8bec40b79698c8)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:43.350838 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:43.585326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2889ebfeff4ef98f8bec40b79698c8', 'request_id': 'req-4c51a378-f6c4-46af-9cd7-ae01525a0648'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:43.804164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2889ebfeff4ef98f8bec40b79698c8', 'request_id': 'req-4c51a378-f6c4-46af-9cd7-ae01525a0648'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:43.810315 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2889ebfeff4ef98f8bec40b79698c8', 'request_id': 'req-4c51a378-f6c4-46af-9cd7-ae01525a0648'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:43.822025 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] The token cache is being invalidate because project 0c2889ebfeff4ef98f8bec40b79698c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:43.822315 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:43.822938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4c51a378-f6c4-46af-9cd7-ae01525a0648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:43.824521 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1250/2514] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:43 2022] DELETE /identity/v3/projects/0c2889ebfeff4ef98f8bec40b79698c8 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:43.828654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:43.830794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:43.843767 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c4fc1-dff2-434b-80b6-1cc744de6fed admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:43.900088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:43.900738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:43.901015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:43.901280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] PATH_INFO: `/v3/users/0c9c02792bba494dac9fc496b889f9b0` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:43.902447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9c02792bba494dac9fc496b889f9b0)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:43.904306 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:44.144178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:44.144178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:44.144178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:44.144178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] MFA Rules not processed for user `8df40cd433e240f9b6d3f24b971ffdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:44.144178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:44.186952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1251/2515] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:07:44.385714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9c02792bba494dac9fc496b889f9b0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:44.393584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9c02792bba494dac9fc496b889f9b0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:44.394339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9c02792bba494dac9fc496b889f9b0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:44.394973 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9c02792bba494dac9fc496b889f9b0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:44.601414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e0f2f2b5-6c36-49c3-bcb1-55198dea8463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9c02792bba494dac9fc496b889f9b0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:44.619979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1265/2516] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:07:43 2022] DELETE /identity/v3/users/0c9c02792bba494dac9fc496b889f9b0 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:45.192827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:45.192827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:45.195265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:45.201102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08320ae-ecdd-494b-9be5-c7cb7bb7c9fd None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:45.257445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:45.258088 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:45.258366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:45.258639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] PATH_INFO: `/v3/projects/64b1715258cf41f6aa0dadfc9ff29164` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:45.259770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b1715258cf41f6aa0dadfc9ff29164)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:45.261737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:45.555405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b1715258cf41f6aa0dadfc9ff29164', 'request_id': 'req-0d4cb36d-b58a-4221-a143-8f770d032a26'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:45.738180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b1715258cf41f6aa0dadfc9ff29164', 'request_id': 'req-0d4cb36d-b58a-4221-a143-8f770d032a26'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:45.744494 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b1715258cf41f6aa0dadfc9ff29164', 'request_id': 'req-0d4cb36d-b58a-4221-a143-8f770d032a26'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:45.756434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] The token cache is being invalidate because project 64b1715258cf41f6aa0dadfc9ff29164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:07:45.756726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:45.757361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0d4cb36d-b58a-4221-a143-8f770d032a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:07:45.758905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1252/2517] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:07:45 2022] DELETE /identity/v3/projects/64b1715258cf41f6aa0dadfc9ff29164 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:45.763704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:45.764201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:45.764297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:45.784116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:45.785283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:45.789747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:45.847352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1266/2518] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:45.855111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:45.855396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:45.855666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:45.876525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:45.877716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:45.885504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1253/2519] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:45.889600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:45.900345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7182ce1-4cee-4005-84f3-cfa28308ecd1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:45.904938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:45.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:45.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:45.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:45.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:07:45.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:45.917406 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:45.921831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1267/2520] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:07:45.926547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:45.936049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dac42-4cd2-4b22-9fb9-27090f5cacd5 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:45.941444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:45.942139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:45.942518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:45.942855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:45.943560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:45.945551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:46.423621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1254/2521] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:07:45 2022] POST /identity/v3/projects => generated 369 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:07:46.427982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:46.438040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a5d85-a810-45bc-a87f-7630340a825a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:46.493993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:46.494600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:46.494882 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:46.495154 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:46.495799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:46.497706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:46.697403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1268/2522] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:07:46 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:07:46.701766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:46.714555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba73d10-e273-4f09-b104-ae6ad486fd0f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:46.720740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:46.721538 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:46.721858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:46.722208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:46.722868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:46.724965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:46.735535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1255/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:46.739725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:46.750032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e44df-f9b5-436c-8f15-e046bf2b86af admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:46.758264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032fda-997e-4ca9-9055-7f45c6d7931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:46.758264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032fda-997e-4ca9-9055-7f45c6d7931a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:46.758264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032fda-997e-4ca9-9055-7f45c6d7931a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:46.758264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032fda-997e-4ca9-9055-7f45c6d7931a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:46.758658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032fda-997e-4ca9-9055-7f45c6d7931a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:46.758658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032fda-997e-4ca9-9055-7f45c6d7931a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:46.768891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1269/2524] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:07:46.772936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:46.784112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ca89-3515-439e-a52e-660f6165726a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:46.788938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:46.789667 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:46.790007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:46.790313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] PATH_INFO: `/v3/projects/420bab5dfc3041f585ebe10fdb4ef71e/users/00db6a1cf6f04c6faabaed45c602bc35/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:46.805572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=420bab5dfc3041f585ebe10fdb4ef71e, user_id=00db6a1cf6f04c6faabaed45c602bc35, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:46.808287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:46.995739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1256/2525] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:07:46 2022] PUT /identity/v3/projects/420bab5dfc3041f585ebe10fdb4ef71e/users/00db6a1cf6f04c6faabaed45c602bc35/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:07:47.008354 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7a45e-5553-4e03-9a2a-9bf53ef89bf9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:47.008949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7a45e-5553-4e03-9a2a-9bf53ef89bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:47.009258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7a45e-5553-4e03-9a2a-9bf53ef89bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:47.039025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b8f7a45e-5553-4e03-9a2a-9bf53ef89bf9 None None] MFA Rules not processed for user `00db6a1cf6f04c6faabaed45c602bc35`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:07:47.039829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b8f7a45e-5553-4e03-9a2a-9bf53ef89bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:47.105081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1270/2526] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:07:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:07:47.119601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:07:47.121422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:07:47.179744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1202e0f-dac8-43bc-9998-7b3c0e6443f1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:07:47.183371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:07:47.184300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:07:47.184635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:07:47.184851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:07:47.194176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:07:47.195607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:07:47.205427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1257/2527] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:07:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:07:47.645285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:57.821723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:07:58.915450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=13 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:14.053087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9f9d-e07d-46d8-8085-0e2f233ce4b2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:14.053592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9f9d-e07d-46d8-8085-0e2f233ce4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:14.053704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9f9d-e07d-46d8-8085-0e2f233ce4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:14.073376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ef0d9f9d-e07d-46d8-8085-0e2f233ce4b2 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:14.076029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ef0d9f9d-e07d-46d8-8085-0e2f233ce4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:14.130508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1271/2528] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:08:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:14.181569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:14.183195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:14.265491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc07238-ec1c-4cf5-afa9-da0750a0c9b3 service glance] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:14.270585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:14.271192 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:14.271466 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:14.271735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:14.279672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:14.281265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:14.289671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1258/2529] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:08:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:08:15.273427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9df77c-4978-4429-9ffc-2dee8a935333 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:15.273867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9df77c-4978-4429-9ffc-2dee8a935333 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:15.273991 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9df77c-4978-4429-9ffc-2dee8a935333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:15.294474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fd9df77c-4978-4429-9ffc-2dee8a935333 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:15.295669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fd9df77c-4978-4429-9ffc-2dee8a935333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:15.303589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1272/2530] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 1 18:08:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:15.316370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:15.326870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff775b-2b05-482b-842d-6a10a75fd3b4 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:15.331971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:15.332587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:15.332868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:15.333139 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:15.341154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:15.342816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:15.351313 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1259/2531] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:08:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:08:16.012510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f38936-b2a8-4c60-8d9f-801f46662955 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:16.012888 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f38936-b2a8-4c60-8d9f-801f46662955 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:16.013169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f38936-b2a8-4c60-8d9f-801f46662955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:16.033867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-78f38936-b2a8-4c60-8d9f-801f46662955 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:16.035067 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-78f38936-b2a8-4c60-8d9f-801f46662955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:16.042284 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1273/2532] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:16.054744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:16.065398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e5158-d413-4271-b1f0-eca17f366c44 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:16.070401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:16.070988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:16.071268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:16.071543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:16.079757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:16.081365 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:16.089268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1260/2533] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:08:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 18:08:16.658510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:16.659238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:16.659590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:16.682750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:16.684621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:16.694096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1274/2534] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 1 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:16.706078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:16.719374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc16226-5a7c-4409-99c1-da997b162237 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:16.725846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958aee69-0883-4950-8d58-8c0816ec379a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:16.726529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aee69-0883-4950-8d58-8c0816ec379a service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:16.726735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aee69-0883-4950-8d58-8c0816ec379a service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:16.727015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aee69-0883-4950-8d58-8c0816ec379a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:16.737668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958aee69-0883-4950-8d58-8c0816ec379a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:16.739842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958aee69-0883-4950-8d58-8c0816ec379a service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:16.753467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1261/2535] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:08:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:08:19.797446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:19.799062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:19.862910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95d4b-fbb7-4edc-8c55-bf82d121ee3b None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:19.867487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:19.868110 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:19.868434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:19.868707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] PATH_INFO: `/v3/users/8df40cd433e240f9b6d3f24b971ffdf8` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:19.869833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df40cd433e240f9b6d3f24b971ffdf8)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:19.871752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:20.070825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df40cd433e240f9b6d3f24b971ffdf8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:20.077404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df40cd433e240f9b6d3f24b971ffdf8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:20.078057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df40cd433e240f9b6d3f24b971ffdf8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:20.078694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df40cd433e240f9b6d3f24b971ffdf8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:20.183104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df40cd433e240f9b6d3f24b971ffdf8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:20.201013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1275/2536] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:08:19 2022] DELETE /identity/v3/users/8df40cd433e240f9b6d3f24b971ffdf8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:08:20.205734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:20.206209 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:20.206304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:20.225181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:20.226425 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:20.291581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1262/2537] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:08:20.301209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:20.302606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:20.309148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:20.363628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e705bb-f244-4216-a351-925687bd7c31 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:20.367728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66694c5a-1d4d-4a42-85ec-a7a7f0d7ac4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:20.368374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66694c5a-1d4d-4a42-85ec-a7a7f0d7ac4c service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:20.368656 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66694c5a-1d4d-4a42-85ec-a7a7f0d7ac4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:20.368979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66694c5a-1d4d-4a42-85ec-a7a7f0d7ac4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:20.380865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66694c5a-1d4d-4a42-85ec-a7a7f0d7ac4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:20.381634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66694c5a-1d4d-4a42-85ec-a7a7f0d7ac4c service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:20.393866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1276/2538] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:08:20.923468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:20.925200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:20.955890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e87521-3d86-4ef7-a8d8-533f9117f57a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:20.963983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:20.964787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:20.965055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:20.965878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] PATH_INFO: `/v3/projects/bc76188669cb4e6a9549c64de29472ca` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:20.967127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc76188669cb4e6a9549c64de29472ca)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:20.969789 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.139126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:21.141962 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc76188669cb4e6a9549c64de29472ca', 'request_id': 'req-42d11f5d-ec91-4c02-b12f-a210f64fd303'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:21.232257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc76188669cb4e6a9549c64de29472ca', 'request_id': 'req-42d11f5d-ec91-4c02-b12f-a210f64fd303'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:21.243710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc76188669cb4e6a9549c64de29472ca', 'request_id': 'req-42d11f5d-ec91-4c02-b12f-a210f64fd303'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:21.257177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] The token cache is being invalidate because project bc76188669cb4e6a9549c64de29472ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:08:21.257481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:21.258135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-42d11f5d-ec91-4c02-b12f-a210f64fd303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:21.259851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1263/2539] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:08:20 2022] DELETE /identity/v3/projects/bc76188669cb4e6a9549c64de29472ca => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:08:21.264727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.265176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.265297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.285598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:21.286792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:21.345605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1277/2540] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:21.354576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.354861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.355135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.376182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:21.377520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:21.385458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1264/2541] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:08:21.389804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:21.399450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15ea1-ffea-49be-9f8d-d8cbb65cc847 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:21.404084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:21.404748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.405016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.421018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.421018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:08:21.421018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:21.421018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.421018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1278/2542] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:08:21.425389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:21.434792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf9f0a-52d7-4595-a2af-7a416d00da64 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:21.439668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:21.440263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.440572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.440844 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.441486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:21.444967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.580122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1265/2543] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:08:21 2022] POST /identity/v3/projects => generated 399 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:08:21.584827 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:21.597285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959d226-0fa1-4411-9ddf-0dae41be7fc8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:21.668409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:21.669030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.669308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.669582 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.670214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:21.672109 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.802190 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1279/2544] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:08:21 2022] POST /identity/v3/users => generated 300 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:08:21.804587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:21.815633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176df73-b8fc-4d0b-a13a-b998f4cc09cc admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:21.821839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:21.822447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.822928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.823246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.824046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:21.826851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.837686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1266/2545] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:08:21.841889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:21.852980 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3359a73-538d-4cbf-a973-be920a95f062 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:21.857788 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcf5e5-e081-4625-aa70-1a713ce8fd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:21.858387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcf5e5-e081-4625-aa70-1a713ce8fd59 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.858664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcf5e5-e081-4625-aa70-1a713ce8fd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.858934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcf5e5-e081-4625-aa70-1a713ce8fd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.859583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcf5e5-e081-4625-aa70-1a713ce8fd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:21.861730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcf5e5-e081-4625-aa70-1a713ce8fd59 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.872454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1280/2546] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:08:21.877160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:21.887270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b7f25-2a4f-43cb-b775-ae7a8b8e3301 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:21.892023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:21.892868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:21.893145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:21.893417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] PATH_INFO: `/v3/projects/fdd702c638f64de0a881600f382cf596/users/cc31edd316e24d018412587be958e4cf/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:21.909112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd702c638f64de0a881600f382cf596, user_id=cc31edd316e24d018412587be958e4cf, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:21.911733 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:21.998082 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1267/2547] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:08:21 2022] PUT /identity/v3/projects/fdd702c638f64de0a881600f382cf596/users/cc31edd316e24d018412587be958e4cf/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:08:22.011686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:22.011997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:22.012274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:22.037970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:22.040779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] MFA Rules not processed for user `cc31edd316e24d018412587be958e4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:22.041998 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:22.104163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1281/2548] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:08:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:22.116732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:22.118501 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:22.176652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f03dd-9266-4230-a1fd-680ef0cbaf5d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:22.180293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:22.180970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:22.181255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:22.181529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:22.190286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:22.191712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:22.200639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1268/2549] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:08:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:08:32.196493 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:32.426364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=26 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:47.892814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:47.894204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:47.950037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c959738-da41-433c-8f6b-ee6690cefe8f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:47.956105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:47.956752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:47.957030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:47.957301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:47.959076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:48.027500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:48.028952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572430c-db13-4056-98de-be6d6d6d4d4c service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:48.040792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1282/2550] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:08:49.760000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:49.761525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:49.773754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ac5eb-bc63-4485-9335-7e7d5c940878 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:49.777420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:49.784879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:49.784879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:49.784879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:49.784879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:49.794534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:49.795971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:49.807797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1269/2551] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:08:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:08:54.633421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:54.633893 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:54.634211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:54.656042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:54.657934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:54.737173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1283/2552] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:08:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:54.751833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:54.776740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:54.833096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e2108-3e2a-468a-b715-5d323e815246 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:54.840374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:54.840374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:54.840686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:54.841165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:54.860800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:54.862400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:54.876978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1270/2553] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:08:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 18:08:54.889173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:54.900169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454388a-0475-45bb-8c7d-1b19e855f0e3 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:54.908938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:54.908938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:54.908938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:54.908938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:54.910223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:54.964001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:54.982565 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:54.983987 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444a5a6-67f8-4eed-ab81-ef12f8851e73 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:54.994444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1284/2554] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:08:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:08:55.371081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:55.466345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d4fec-f3f4-4e1e-9a88-bcd2f8472a37 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:55.466345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:55.466345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:55.466345 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:55.466828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] PATH_INFO: `/v3/users/cc31edd316e24d018412587be958e4cf` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:55.467874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc31edd316e24d018412587be958e4cf)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:55.470616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:55.752389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc31edd316e24d018412587be958e4cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:55.760141 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc31edd316e24d018412587be958e4cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:55.873961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc31edd316e24d018412587be958e4cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:55.875650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc31edd316e24d018412587be958e4cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:55.895787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:08:55.898379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc31edd316e24d018412587be958e4cf'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:55.913366 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1271/2555] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:08:55 2022] DELETE /identity/v3/users/cc31edd316e24d018412587be958e4cf => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:08:55.914845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:55.915332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:55.915437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:55.942339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:08:55.944474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:56.024040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1285/2556] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:08:56.060101 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:56.122258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:56.216435 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a5f80-7948-4253-b7d6-d0338ebbd0f5 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:56.220508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:56.221682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:56.222136 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:56.222575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:56.240824 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:56.242528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:56.262970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1272/2557] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:08:57.273373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:08:57.275201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:08:57.300709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ebbeb-748d-4c0e-b5d1-9347e8cde0bb None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:08:57.327312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:08:57.328779 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:08:57.329011 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:08:57.329289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] PATH_INFO: `/v3/projects/fdd702c638f64de0a881600f382cf596` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:08:57.331674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd702c638f64de0a881600f382cf596)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:08:57.335904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:08:57.725595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd702c638f64de0a881600f382cf596', 'request_id': 'req-994653d2-ba97-4680-98bc-f33ab79c0304'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:57.845583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd702c638f64de0a881600f382cf596', 'request_id': 'req-994653d2-ba97-4680-98bc-f33ab79c0304'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:57.881789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd702c638f64de0a881600f382cf596', 'request_id': 'req-994653d2-ba97-4680-98bc-f33ab79c0304'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:57.881789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] The token cache is being invalidate because project fdd702c638f64de0a881600f382cf596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:08:57.881789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:57.881789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:08:57.882112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1286/2558] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:08:57 2022] DELETE /identity/v3/projects/fdd702c638f64de0a881600f382cf596 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:09:01.754027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:01.755534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:01.839731 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d588ef-5f16-44c4-a321-afbd23d4ab3f service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:01.933469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:01.934214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:01.934543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:01.934820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:01.936686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:01.996843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:02.027937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:02.029627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:02.044348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1273/2559] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:07.871079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:12.616152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=12 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:16.733268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:16.735339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:16.801388 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994653d2-ba97-4680-98bc-f33ab79c0304 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:16.806074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:16.806701 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:16.806981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:16.807259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] PATH_INFO: `/v3/users/00db6a1cf6f04c6faabaed45c602bc35` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:16.808451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] RBAC: Authorizing `identity:delete_user(user_id=00db6a1cf6f04c6faabaed45c602bc35)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:16.810387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:17.167840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00db6a1cf6f04c6faabaed45c602bc35'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:17.175201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00db6a1cf6f04c6faabaed45c602bc35'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:17.176103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00db6a1cf6f04c6faabaed45c602bc35'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:17.176783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00db6a1cf6f04c6faabaed45c602bc35'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:17.270071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:17.271588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:17.299301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-616bf349-091a-40d8-9d3f-6925289774df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00db6a1cf6f04c6faabaed45c602bc35'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:17.318419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1287/2560] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:09:16 2022] DELETE /identity/v3/users/00db6a1cf6f04c6faabaed45c602bc35 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:09:17.322537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc4369-072a-49fb-ba0d-0fef2c9d067e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:17.323073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc4369-072a-49fb-ba0d-0fef2c9d067e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:17.323437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc4369-072a-49fb-ba0d-0fef2c9d067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:17.332164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89303ebf-f90c-400a-a57d-2017e15baead service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:17.344240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c8fc4369-072a-49fb-ba0d-0fef2c9d067e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:17.345689 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c8fc4369-072a-49fb-ba0d-0fef2c9d067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:17.386166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:17.386912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:17.387195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:17.387465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:17.389277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:17.399685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:17.401146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:17.410144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1274/2561] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:09:17.410937 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1288/2562] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:09:17.420403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:17.421837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:17.432586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c15a5-22f9-4948-b397-f404ebd7c460 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:17.436523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705095bd-59bf-45ae-b6f5-a12bdd954cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:17.437162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705095bd-59bf-45ae-b6f5-a12bdd954cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:17.437516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705095bd-59bf-45ae-b6f5-a12bdd954cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:17.437790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705095bd-59bf-45ae-b6f5-a12bdd954cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:17.447442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705095bd-59bf-45ae-b6f5-a12bdd954cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:17.448895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705095bd-59bf-45ae-b6f5-a12bdd954cd7 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:17.463874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1275/2563] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:17.903345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a91a2-2474-4dbc-ae75-224db6764bc9 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:17.904127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a91a2-2474-4dbc-ae75-224db6764bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:17.904127 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a91a2-2474-4dbc-ae75-224db6764bc9 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:17.905670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1289/2564] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:09:17.912613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:17.912897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:17.913168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:17.939489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:17.941603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:17.989433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a91a2-2474-4dbc-ae75-224db6764bc9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:17.992595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ff5a91a2-2474-4dbc-ae75-224db6764bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.021924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a91a2-2474-4dbc-ae75-224db6764bc9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.030968 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.031889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.032125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.032442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] PATH_INFO: `/v3/projects/420bab5dfc3041f585ebe10fdb4ef71e` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.034057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420bab5dfc3041f585ebe10fdb4ef71e)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.036185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.052886 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1276/2565] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:09:18.056412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:18.069814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.071236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.114233 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=100 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:18.130219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e110562-156f-4200-a67d-34f57e5c46db None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.134006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d003f1-ebb4-4166-888f-815d0f5c0430 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.134620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d003f1-ebb4-4166-888f-815d0f5c0430 service glance] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.134901 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d003f1-ebb4-4166-888f-815d0f5c0430 service glance] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.135171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d003f1-ebb4-4166-888f-815d0f5c0430 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.146813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d003f1-ebb4-4166-888f-815d0f5c0430 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.148258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d003f1-ebb4-4166-888f-815d0f5c0430 service glance] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.159571 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420bab5dfc3041f585ebe10fdb4ef71e', 'request_id': 'req-01647285-11ab-426c-bfbe-b908aaa63ba8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:18.161060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1277/2566] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:09:18.251721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420bab5dfc3041f585ebe10fdb4ef71e', 'request_id': 'req-01647285-11ab-426c-bfbe-b908aaa63ba8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:18.259899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420bab5dfc3041f585ebe10fdb4ef71e', 'request_id': 'req-01647285-11ab-426c-bfbe-b908aaa63ba8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:18.274755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] The token cache is being invalidate because project 420bab5dfc3041f585ebe10fdb4ef71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:09:18.275108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:18.276140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-01647285-11ab-426c-bfbe-b908aaa63ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:09:18.278128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1290/2567] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:09:17 2022] DELETE /identity/v3/projects/420bab5dfc3041f585ebe10fdb4ef71e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:09:18.284105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010f933-62be-4b1b-b4fb-7e76cc914b6e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.284677 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010f933-62be-4b1b-b4fb-7e76cc914b6e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.284811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010f933-62be-4b1b-b4fb-7e76cc914b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.306857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-7010f933-62be-4b1b-b4fb-7e76cc914b6e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:18.307992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7010f933-62be-4b1b-b4fb-7e76cc914b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.337900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6966a0-e177-4327-87ae-af2625e1a79c None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.338185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6966a0-e177-4327-87ae-af2625e1a79c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.338561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6966a0-e177-4327-87ae-af2625e1a79c None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.340493 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1291/2568] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 18:09:18.346217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.346599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.346874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.371183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:18.372437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.380249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1278/2569] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:09:18.389076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.389585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.390162 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.413108 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:18.414317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.425700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1279/2570] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:09:18.430010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.437397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1292/2571] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:09:18.439250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5588-0847-45f2-bd5f-c4db0cf47516 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.444847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.445124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.445400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.445701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.446358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:09:18.446719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.448986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.451151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.452742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.464157 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abda78-3672-4242-afde-e7a80f6d1c8d None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.464953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1280/2572] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:09:18.468793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104fb1d-bab2-47e6-9a54-ddf20ed97b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.469424 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104fb1d-bab2-47e6-9a54-ddf20ed97b05 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.469699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104fb1d-bab2-47e6-9a54-ddf20ed97b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.470073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104fb1d-bab2-47e6-9a54-ddf20ed97b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.471597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.479383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104fb1d-bab2-47e6-9a54-ddf20ed97b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.480852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104fb1d-bab2-47e6-9a54-ddf20ed97b05 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.482364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0797-b39c-48d9-b67c-9e4462317360 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.487786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.488418 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.488699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.489000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.489638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.491536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.491971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1293/2573] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:09:18.573348 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be0c1-8ec5-42d3-8903-7296709aafdd None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.573637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be0c1-8ec5-42d3-8903-7296709aafdd None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.573939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be0c1-8ec5-42d3-8903-7296709aafdd None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.575357 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1294/2574] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 18:09:18.579959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.580239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.580533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.601866 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:18.603928 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.610644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1281/2575] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:09:18.611771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.674681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11464-1406-40b6-96d6-a836bc2b1522 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.677713 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1295/2576] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:09:18.679653 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.680268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.680605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.680884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.681524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.683447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.688809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.722417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:18.745261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261eafc-798b-4b7d-8b37-f809e03dfa28 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.749723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.750335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.750609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.750884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.761518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.762820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.776101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1296/2578] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:09:18.776371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1282/2578] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/users => generated 282 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:09:18.777161 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.788105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30385b-d48c-4a10-9f5d-0baaa9cc47d4 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.792867 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e5af4-daff-44bc-affa-73cf0ced8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.793476 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e5af4-daff-44bc-affa-73cf0ced8d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.793756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e5af4-daff-44bc-affa-73cf0ced8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.794027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e5af4-daff-44bc-affa-73cf0ced8d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.794673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e5af4-daff-44bc-affa-73cf0ced8d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.796838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e5af4-daff-44bc-affa-73cf0ced8d76 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.806949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1297/2579] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:09:18.814595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.823350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe7b6-2e90-4d30-9400-d45f566c69e1 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.828846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.829563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.829913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.830260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.831123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.833050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.850603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1283/2580] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:09:18.851638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.852004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52bf8-dd21-4860-9e47-83d9b309a034 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.852394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52bf8-dd21-4860-9e47-83d9b309a034 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.852745 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52bf8-dd21-4860-9e47-83d9b309a034 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.854551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1298/2581] 10.222.0.15 () {54 vars in 961 bytes} [Thu Sep 1 18:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:09:18.860672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.861042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.862053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.863674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ceee5-4149-4e52-9b26-802d95bb9cc7 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.868860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d54ad-c9ab-4b90-ab9f-b4c0ac0cc0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.870175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d54ad-c9ab-4b90-ab9f-b4c0ac0cc0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.870297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d54ad-c9ab-4b90-ab9f-b4c0ac0cc0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.870572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d54ad-c9ab-4b90-ab9f-b4c0ac0cc0f5 admin admin] PATH_INFO: `/v3/projects/5d9be9378fcf493ebfab860ad8f8d307/users/c9f9177bd7d74e2f83b0a251771d8436/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.886822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] MFA Rules not processed for user `4c88695a14384a0b85bb20a0589692bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:18.887728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d54ad-c9ab-4b90-ab9f-b4c0ac0cc0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9be9378fcf493ebfab860ad8f8d307, user_id=c9f9177bd7d74e2f83b0a251771d8436, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.888055 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.891158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d54ad-c9ab-4b90-ab9f-b4c0ac0cc0f5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.893860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:18.952716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1299/2582] 10.222.0.15 () {58 vars in 1058 bytes} [Thu Sep 1 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:09:18.960289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:18.970339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b7b32-2145-4a8a-a597-e3ae35f9866a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:18.975741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:18.976360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:18.976646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:18.976942 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:18.978777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:18.991153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:18.992778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:18.993605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1284/2583] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:09:18 2022] PUT /identity/v3/projects/5d9be9378fcf493ebfab860ad8f8d307/users/c9f9177bd7d74e2f83b0a251771d8436/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:09:19.004644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f5e39-2d4d-4e63-bdbd-c01322100054 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:19.004930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f5e39-2d4d-4e63-bdbd-c01322100054 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:19.005205 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f5e39-2d4d-4e63-bdbd-c01322100054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:19.005378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1300/2584] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:09:19.035390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-710f5e39-2d4d-4e63-bdbd-c01322100054 None None] MFA Rules not processed for user `c9f9177bd7d74e2f83b0a251771d8436`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:19.036614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-710f5e39-2d4d-4e63-bdbd-c01322100054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:19.119607 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1285/2585] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:09:19.127147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:19.128633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:19.192281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a774-b25d-4d13-ab93-3ed67123785b service placement] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:19.241920 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:19.242531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:19.242841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:19.243111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:19.251795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:19.253258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:19.262313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1301/2586] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:21.852467 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:21.852954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:21.853063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:21.872997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] MFA Rules not processed for user `6201e7d144874370b4da432a529014d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:21.874245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:21.931488 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1286/2587] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:09:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:09:22.206680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:22.206680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:22.206680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:22.216908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd7ff5-3f5b-4431-83bf-3b5bb87c665f service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:22.221104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:22.221700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:22.221977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:22.222246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:22.223959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:22.233313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:22.235342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362b0a3-83c3-4c69-967f-e7067a9c7d4a service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:22.243732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1302/2588] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:24.440957 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:24.451277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca218b8-ae75-441d-a6bb-83afbb48213e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:24.455021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:24.455676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:24.455958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:24.456228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:24.458211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:24.469117 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:24.470582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:24.479393 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1287/2589] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:25.654019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:25.654549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:25.654549 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:25.678506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:25.679798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:25.749513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1303/2590] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:09:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:09:25.772916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:25.774320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:25.850432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd5495-ae52-4a89-889e-ac5213b015d6 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:25.855685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:25.857217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:25.857217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:25.857573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:25.867811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:25.869740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:25.878688 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1288/2591] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:09:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:09:35.742710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=49 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:35.875848 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:INSERT=1,keystone:SELECT=97 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:44.412592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:44.414035 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:44.487554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba83ee-0276-4bbf-817e-88aa5e919aac None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:44.493183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:44.493792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:44.494075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:44.494350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:44.496091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:44.508750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:44.510356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a13bb6-9f32-46d0-bda3-0bdfbb45488d service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:44.521696 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1304/2592] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:52.297682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:52.309816 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9aede-19af-4ad9-a4b2-c4b9ebccf6ce service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:52.316256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:52.316841 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:52.317116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:52.317391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:52.319309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:52.333049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:52.340478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:52.353009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1289/2593] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:09:54.517327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=34 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:09:59.279966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:59.281224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:59.281224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:59.307203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:09:59.307203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:09:59.310164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1305/2594] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 1 18:09:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:09:59.322424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:09:59.334093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e016d9-deb4-42e3-b61c-15cc477c8061 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:09:59.339003 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:09:59.339596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:09:59.339870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:09:59.340137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:09:59.348779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:09:59.350338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:09:59.358439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1290/2595] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:09:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:10:09.295310 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:10:09.354940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=12 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:10:16.433513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:10:16.434910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:10:16.450433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da17a4-38ab-404d-b811-693d1d5faeac None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:10:16.456829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:10:16.457433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:10:16.457709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:10:16.457982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:10:16.459762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:10:16.517382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:10:16.518941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4998f7-7b58-4377-882c-141e0d8ec6f0 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:10:16.527930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1306/2596] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:10:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:10:26.523670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=28 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:10:36.647092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:10:36.708473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1e1c2-923c-4095-af82-6d9095586250 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:10:36.713433 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:10:36.714071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:10:36.714346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:10:36.714700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] PATH_INFO: `/v3/users/c9f9177bd7d74e2f83b0a251771d8436` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:10:36.715835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f9177bd7d74e2f83b0a251771d8436)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:10:36.718092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:10:36.967599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f9177bd7d74e2f83b0a251771d8436'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:36.974812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f9177bd7d74e2f83b0a251771d8436'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:37.058246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f9177bd7d74e2f83b0a251771d8436'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:37.059060 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f9177bd7d74e2f83b0a251771d8436'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:37.072869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f9177bd7d74e2f83b0a251771d8436'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:37.079537 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1291/2597] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:10:36 2022] DELETE /identity/v3/users/c9f9177bd7d74e2f83b0a251771d8436 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:10:37.084207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:10:37.084774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:10:37.084774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:10:37.104251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:10:37.105511 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:10:37.174776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1307/2598] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:10:37.184631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:10:37.186087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:10:37.248963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cbcbd-a1eb-4f4a-b534-2422635d0a87 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:10:37.252907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45631e6c-a67c-4b57-a352-e4fd64116027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:10:37.253519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45631e6c-a67c-4b57-a352-e4fd64116027 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:10:37.253797 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45631e6c-a67c-4b57-a352-e4fd64116027 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:10:37.254064 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45631e6c-a67c-4b57-a352-e4fd64116027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:10:37.264281 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45631e6c-a67c-4b57-a352-e4fd64116027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:10:37.265935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45631e6c-a67c-4b57-a352-e4fd64116027 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:10:37.275810 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1292/2599] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:10:37.747025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:10:37.752919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:10:37.761228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd74d0-11a1-414b-ac85-0653a8ece281 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:10:37.766558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:10:37.767195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:10:37.767470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:10:37.767762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] PATH_INFO: `/v3/projects/5d9be9378fcf493ebfab860ad8f8d307` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:10:37.769020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9be9378fcf493ebfab860ad8f8d307)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:10:37.770758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:10:37.934251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9be9378fcf493ebfab860ad8f8d307', 'request_id': 'req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:38.031301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9be9378fcf493ebfab860ad8f8d307', 'request_id': 'req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:38.040186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9be9378fcf493ebfab860ad8f8d307', 'request_id': 'req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:38.059843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] The token cache is being invalidate because project 5d9be9378fcf493ebfab860ad8f8d307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:10:38.060239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:38.061637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:10:38.063175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1308/2600] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:10:37 2022] DELETE /identity/v3/projects/5d9be9378fcf493ebfab860ad8f8d307 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:10:47.271575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:10:48.056899 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:10:51.322308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39577734-8988-4864-9b49-889a8b309eba None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:10:51.322830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39577734-8988-4864-9b49-889a8b309eba None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:10:51.322945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39577734-8988-4864-9b49-889a8b309eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:10:51.342515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-39577734-8988-4864-9b49-889a8b309eba None None] MFA Rules not processed for user `01f1499bbf89441dae30433aaf1d29a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:10:51.343642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-39577734-8988-4864-9b49-889a8b309eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:10:51.397270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1293/2601] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:10:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:11:01.390542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=25 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:11:07.910214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:11:07.911839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:07.986061 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b8d8a-4d1e-46e9-9e23-8eb3293864fd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:11:07.991508 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:11:07.996693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:11:07.996693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:11:07.996693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:11:07.996693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:08.052823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:11:08.054433 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:11:08.065256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1309/2602] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:11:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:11:11.425820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39577734-8988-4864-9b49-889a8b309eba None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:11:11.427378 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-39577734-8988-4864-9b49-889a8b309eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:11.499660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39577734-8988-4864-9b49-889a8b309eba None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:11:11.504632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:11:11.505247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:11:11.505524 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:11:11.505791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:11:11.507556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:11.520122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:11:11.521720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:11:11.531984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1294/2603] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:11:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:11:11.599535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:11:11.602028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:11.677688 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eee32-0276-417d-9b7a-1e57f002c726 service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:11:11.682363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:11:11.683148 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:11:11.683450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:11:11.683723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] PATH_INFO: `/v3/users/6201e7d144874370b4da432a529014d2` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:11:11.685462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6201e7d144874370b4da432a529014d2)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:11:11.687367 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:11:11.935195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:11:11.941060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6201e7d144874370b4da432a529014d2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:11.948256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6201e7d144874370b4da432a529014d2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:11.949211 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6201e7d144874370b4da432a529014d2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:11.949881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6201e7d144874370b4da432a529014d2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.058199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5c39373f-9e3e-4751-aeaa-332cd8b910c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6201e7d144874370b4da432a529014d2'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.082727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1310/2604] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:11:11 2022] DELETE /identity/v3/users/6201e7d144874370b4da432a529014d2 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:11:12.350229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:11:12.365230 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e63b7-2670-4df6-bee5-f4075b0c9ad8 service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:11:12.445237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:11:12.445882 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:11:12.446240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:11:12.446587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] PATH_INFO: `/v3/projects/ab11fffacccb4f248d76d99553bdb2a9` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:11:12.447812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab11fffacccb4f248d76d99553bdb2a9)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:11:12.450118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:11:12.588561 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab11fffacccb4f248d76d99553bdb2a9', 'request_id': 'req-85ee4cea-59f7-441b-9ba8-a4915bf9630c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.663615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab11fffacccb4f248d76d99553bdb2a9', 'request_id': 'req-85ee4cea-59f7-441b-9ba8-a4915bf9630c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.671976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab11fffacccb4f248d76d99553bdb2a9', 'request_id': 'req-85ee4cea-59f7-441b-9ba8-a4915bf9630c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.693487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] The token cache is being invalidate because project ab11fffacccb4f248d76d99553bdb2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:11:12.694076 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.695966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85ee4cea-59f7-441b-9ba8-a4915bf9630c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:11:12.696478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1295/2605] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:11:12 2022] DELETE /identity/v3/projects/ab11fffacccb4f248d76d99553bdb2a9 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:11:14.509321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:11:14.509768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:11:14.509863 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:11:14.529295 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] MFA Rules not processed for user `14bd04e951254ec5bc29999bceeea214`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:11:14.530496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:14.585247 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1311/2606] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:11:22.689406 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:11:24.578322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=5,keystone:SELECT=39,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:11:46.581781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:11:46.582298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:11:46.582298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:11:46.601411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] MFA Rules not processed for user `fb6a740de8b9417b89afb08f230aa4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:11:46.602625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:11:46.657831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1296/2607] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:11:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:11:56.650135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=25 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:12.925621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:12.927059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:13.067450 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784b1b-6d68-4935-a521-db4cddfce2de None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:13.073415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:13.074047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:13.075526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:13.075526 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:13.077150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:13.089318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:13.091759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:13.101781 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1312/2608] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:12:23.069837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:23.071323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:23.098254 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=63 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:23.212938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ba781-f0ba-443d-ab8d-7ceb582b32e8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:23.220089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:23.220726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:23.221001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:23.221271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:23.223680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:23.291917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:23.293615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:23.302776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1297/2609] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:12:24.766104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:24.767519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:24.842962 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663c569-3a53-4b0d-beeb-d47c9de0523f service placement] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:24.850636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:24.850636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:24.850636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:24.850636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:24.851048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:24.932209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:24.934333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:24.954340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1313/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:12:28.893379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:28.894894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:28.910370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0451f3-170d-4180-9efa-52231b867c8b service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:28.914968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:28.915601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:28.915878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:28.916152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] PATH_INFO: `/v3/users/01f1499bbf89441dae30433aaf1d29a7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:28.917343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f1499bbf89441dae30433aaf1d29a7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:28.919888 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:29.104206 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:29.109961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f1499bbf89441dae30433aaf1d29a7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.116689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f1499bbf89441dae30433aaf1d29a7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.244849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f1499bbf89441dae30433aaf1d29a7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.245697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f1499bbf89441dae30433aaf1d29a7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.260268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d1f7048c-5012-4966-a541-e3eec1161815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f1499bbf89441dae30433aaf1d29a7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.265884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1298/2611] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:12:28 2022] DELETE /identity/v3/users/01f1499bbf89441dae30433aaf1d29a7 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:12:29.567693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:29.578947 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72cfec-4e31-4850-aa87-4e98c21e310b service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:29.637200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:29.637674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:29.637957 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:29.638233 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] PATH_INFO: `/v3/projects/6749f998c37a43428eb7e1233408c560` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:29.639370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6749f998c37a43428eb7e1233408c560)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:29.641302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:29.643576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:29.764995 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6749f998c37a43428eb7e1233408c560', 'request_id': 'req-651089e9-95e5-473b-b141-718330e7505f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.829673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6749f998c37a43428eb7e1233408c560', 'request_id': 'req-651089e9-95e5-473b-b141-718330e7505f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.836280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6749f998c37a43428eb7e1233408c560', 'request_id': 'req-651089e9-95e5-473b-b141-718330e7505f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.849794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] The token cache is being invalidate because project 6749f998c37a43428eb7e1233408c560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:12:29.850086 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.850806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-651089e9-95e5-473b-b141-718330e7505f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:12:29.852326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1314/2612] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:12:29 2022] DELETE /identity/v3/projects/6749f998c37a43428eb7e1233408c560 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:12:29.858178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:29.858692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:29.859659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:29.923487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:29.924737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:29.984280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1299/2613] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:29.994039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:29.994324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:29.994600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.015287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:30.016588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:30.024442 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1315/2614] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:30.028922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.038944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630fef1-1423-49b0-a54e-e2453510f0e9 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.044275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.044899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.045177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.045473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.046118 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:12:30.046417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.047991 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.060057 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1300/2615] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:12:30.064213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.073375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fd7b0-35c5-448f-bf26-2022cd527660 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.078004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.078588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.078864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.079131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.079759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.081677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.167452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1316/2616] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:12:30 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:12:30.172363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.182357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4a409-1717-4214-8695-14c4789820c4 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.239745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.240382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.240660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.240930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.241556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.243804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.331921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1301/2617] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:12:30 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:12:30.336259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.347702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0cc2c-5292-4748-bcb0-5d734854685e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.352336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.353046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.353324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.353596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.354238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.356546 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.366464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1317/2618] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:12:30.370412 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.375710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:30.380971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175edbee-809c-4f68-833b-3a95a9a70131 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.385652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db738-8f2d-4d5c-8dbd-36c9dd52dd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.390284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db738-8f2d-4d5c-8dbd-36c9dd52dd5b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.390284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db738-8f2d-4d5c-8dbd-36c9dd52dd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.390284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db738-8f2d-4d5c-8dbd-36c9dd52dd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.390284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db738-8f2d-4d5c-8dbd-36c9dd52dd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.390284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db738-8f2d-4d5c-8dbd-36c9dd52dd5b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.400986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1302/2619] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:12:30.405738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.416275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6652-e8bf-42ea-9030-6f7cf46140c7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.423004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.423004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.423004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.423323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] PATH_INFO: `/v3/projects/1c789d40f8a84058a49533d382fe5303/users/f747a57b1a604347b9d2828bdddfa0e9/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.437933 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c789d40f8a84058a49533d382fe5303, user_id=f747a57b1a604347b9d2828bdddfa0e9, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.440636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.520296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1318/2620] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:12:30 2022] PUT /identity/v3/projects/1c789d40f8a84058a49533d382fe5303/users/f747a57b1a604347b9d2828bdddfa0e9/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:12:30.525403 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17c4a1-8fcf-46e7-8f5b-0679682ba35f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.525691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17c4a1-8fcf-46e7-8f5b-0679682ba35f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.525966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17c4a1-8fcf-46e7-8f5b-0679682ba35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.545834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5e17c4a1-8fcf-46e7-8f5b-0679682ba35f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:30.547011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5e17c4a1-8fcf-46e7-8f5b-0679682ba35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:30.605121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1303/2621] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:30.614809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:30.616523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:30.675887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834020-fb61-46c3-9b4b-630aaab9480a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:30.679437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:30.680041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:30.680334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:30.680611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:30.690265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:30.691732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:30.701994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1319/2622] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:12:36.423736 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7072a3-baf7-4ee6-be7e-69fe11eae650 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:36.424284 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7072a3-baf7-4ee6-be7e-69fe11eae650 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:36.424905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7072a3-baf7-4ee6-be7e-69fe11eae650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:36.468926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-dd7072a3-baf7-4ee6-be7e-69fe11eae650 None None] MFA Rules not processed for user `f747a57b1a604347b9d2828bdddfa0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:36.470615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dd7072a3-baf7-4ee6-be7e-69fe11eae650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:36.545300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1304/2623] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:12:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:36.559740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:36.561280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:36.598507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:36.641246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80790-c9ea-4335-87ac-693e523415ef service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:36.646897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef80fc-fed9-464e-a4cf-5cf56e591712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:36.647759 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef80fc-fed9-464e-a4cf-5cf56e591712 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:36.648092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef80fc-fed9-464e-a4cf-5cf56e591712 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:36.648405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef80fc-fed9-464e-a4cf-5cf56e591712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:36.657682 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef80fc-fed9-464e-a4cf-5cf56e591712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:36.659434 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef80fc-fed9-464e-a4cf-5cf56e591712 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:36.670536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1320/2624] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:12:39.711255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3065a59-47a4-4f86-b942-ec58a9279328 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:39.712260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3065a59-47a4-4f86-b942-ec58a9279328 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:39.712569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3065a59-47a4-4f86-b942-ec58a9279328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:39.741020 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d3065a59-47a4-4f86-b942-ec58a9279328 None None] MFA Rules not processed for user `f747a57b1a604347b9d2828bdddfa0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:39.742525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d3065a59-47a4-4f86-b942-ec58a9279328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:39.764059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1305/2625] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:12:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:43.922328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eab48-4c2e-41c5-a0cc-b65df4e3ed87 None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:43.923158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eab48-4c2e-41c5-a0cc-b65df4e3ed87 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:43.923158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eab48-4c2e-41c5-a0cc-b65df4e3ed87 None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:43.924386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1321/2626] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:12:43.932843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1acb55-6740-4d07-8737-66a99fb2b9eb None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:43.934124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1acb55-6740-4d07-8737-66a99fb2b9eb None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:43.934427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1acb55-6740-4d07-8737-66a99fb2b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:43.968565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3d1acb55-6740-4d07-8737-66a99fb2b9eb None None] MFA Rules not processed for user `c16d573ee440426aaa09eddb9c2658dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:43.969818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3d1acb55-6740-4d07-8737-66a99fb2b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:43.976515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1306/2627] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:12:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:43.989077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eab48-4c2e-41c5-a0cc-b65df4e3ed87 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:43.994792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dc6eab48-4c2e-41c5-a0cc-b65df4e3ed87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:44.005327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eab48-4c2e-41c5-a0cc-b65df4e3ed87 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:44.008954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2ac73-681e-4b73-921a-3560aebd74e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:44.009551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2ac73-681e-4b73-921a-3560aebd74e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:44.009832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2ac73-681e-4b73-921a-3560aebd74e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:44.010106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2ac73-681e-4b73-921a-3560aebd74e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:44.017815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2ac73-681e-4b73-921a-3560aebd74e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:44.019285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2ac73-681e-4b73-921a-3560aebd74e5 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:44.027308 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1322/2628] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:12:44.109503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d614-dbc2-44f6-9b4c-2248741e48c6 None None] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:44.109787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d614-dbc2-44f6-9b4c-2248741e48c6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:44.110059 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d614-dbc2-44f6-9b4c-2248741e48c6 None None] PATH_INFO: `/` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:44.111427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1307/2629] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:12:44.116326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:44.116628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:44.116903 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:44.138786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:12:44.139901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:44.149025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1323/2630] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:12:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:12:44.159383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d614-dbc2-44f6-9b4c-2248741e48c6 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:44.171454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d614-dbc2-44f6-9b4c-2248741e48c6 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:44.175188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:44.175796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:44.176116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:44.176428 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:44.187974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:44.189461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:44.202595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1308/2631] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:12:54.133049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=29 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:54.197219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=79 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:12:58.064822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:12:58.066429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:58.080822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70281982-4d84-4f6e-90b9-03b5432236ba None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:12:58.088765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:12:58.089410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:12:58.089690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:12:58.090463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:12:58.093370 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:12:58.105918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:12:58.107567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:12:58.118382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1324/2632] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:12:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:13:08.113792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:13:48.577475 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:13:48.586111 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904fc12-f025-4442-b355-aa605e91470d service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:13:48.589740 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:13:48.590327 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:13:48.590603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:13:48.590870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:13:48.592602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:13:48.606459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:13:48.607861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc3338-954a-4faa-8542-099327b0487a service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:13:48.619291 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1309/2633] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:13:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:13:49.857404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:13:49.867240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff29ea-b622-4f7f-ade5-2469c3e94329 service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:13:49.873429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:13:49.873429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:13:49.873429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:13:49.873429 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:13:49.873831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:13:49.886601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:13:49.887993 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80384608-683b-4faf-a29a-7ff9b91173d3 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:13:49.898612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1325/2634] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:13:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:13:50.988664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:13:50.989116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:13:50.989234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:13:51.008846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] MFA Rules not processed for user `14bd04e951254ec5bc29999bceeea214`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:13:51.010038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:13:51.067786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1310/2635] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:13:59.894069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:14:00.618032 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:00.618326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:00.618605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:00.638886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:14:00.640105 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:00.696524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1326/2636] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:14:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:14:00.819219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:00.820676 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:00.888259 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aeedf-934d-47ab-bf4a-afdfbb247940 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:00.893314 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:00.893909 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:00.894185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:00.894458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:00.902207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:00.903761 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:00.912334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1311/2637] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:14:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:14:02.938253 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:02.952545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1de9-186b-4b7a-a6f3-d7b643740ed7 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:02.960121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:02.960768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:02.961047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:02.961599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:02.964131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:03.068364 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:03.069965 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c87d73-a40f-42b2-9ce4-6b38c8810b56 service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:03.090847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1327/2638] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:14:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:14:10.908763 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=63 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:14:13.087768 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=66 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:14:14.366202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:14.375672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8216656-a3f3-4706-831d-b6aa22596787 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:14.379744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:14.380595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:14.380870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:14.381363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:14.383058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:14.460924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:14.462377 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e4144-0915-4f7b-94fc-28811ac12ad9 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:14.473444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1312/2639] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:14:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:14:20.638880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67a9fa-efb6-4698-b19e-873b359e9b2e None None] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:20.644355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67a9fa-efb6-4698-b19e-873b359e9b2e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:20.644355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67a9fa-efb6-4698-b19e-873b359e9b2e None None] PATH_INFO: `/` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:20.644355 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1328/2640] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 1 18:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 18:14:20.646381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:20.646807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:20.646920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:20.686389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] MFA Rules not processed for user `99dec0395ef74fd5a6023ae06a1b9538`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:14:20.687654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:20.698180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1313/2641] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 1 18:14:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:14:20.709403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67a9fa-efb6-4698-b19e-873b359e9b2e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:20.711293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bd67a9fa-efb6-4698-b19e-873b359e9b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:20.770883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67a9fa-efb6-4698-b19e-873b359e9b2e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:20.774439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:20.775045 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:20.775324 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:20.775594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:20.785580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:20.787240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:20.797686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1329/2642] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:14:21.180396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:21.181769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:21.252040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89762d-efde-4725-9e89-a9f4c09903b0 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:21.257239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:21.257843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:21.258120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:21.258388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:21.260149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:21.271846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:21.273439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7879cef-27a0-4549-bec5-b8dc559deca8 service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:21.283543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1314/2643] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:14:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:14:22.996994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:23.006739 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd760f-10f9-4cd7-a105-9b5a48fb5146 service glance] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:23.010337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:23.010945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:23.011223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:23.011492 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:23.013281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:23.023941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:23.025383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:23.034199 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1330/2644] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:14:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:14:23.902664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2e191-27ae-4869-bb2c-2f5ac1873d45 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:23.903195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2e191-27ae-4869-bb2c-2f5ac1873d45 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:23.903195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2e191-27ae-4869-bb2c-2f5ac1873d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:23.958443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5ed2e191-27ae-4869-bb2c-2f5ac1873d45 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:14:23.959851 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5ed2e191-27ae-4869-bb2c-2f5ac1873d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:23.979414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1315/2645] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:14:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:14:23.994706 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:24.017994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aac06-3051-4010-8b78-93e349df8394 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:24.023120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:24.023760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:24.024152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:24.024693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:24.038618 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:24.040200 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:24.049449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1331/2646] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:14:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 18:14:33.976604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:14:34.045446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=42 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:14:46.567330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:46.567822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:46.567936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:46.591294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:14:46.592540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:14:46.600203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1316/2647] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 1 18:14:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:14:46.612381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:14:46.622969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294cddd-2682-4814-9faa-59a6ec0d06d7 service swift] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:14:46.628539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:14:46.629137 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:14:46.629441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:14:46.629716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:14:46.637654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:14:46.639204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:14:46.647760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1332/2648] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:14:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:14:56.585187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:14:56.644677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:15:02.846926 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:02.859920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc2966-6318-4df5-9350-4d7df7505649 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:02.865783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:02.866381 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:02.866661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:02.866932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:02.868673 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:02.884974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:02.886717 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:02.898588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1317/2649] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:15:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:15:06.671251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:06.680752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a3a0-8b4b-42d5-b39e-e2c74e60df1b service swift] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:06.684287 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:06.684908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:06.685191 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:06.685467 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:06.687169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:06.701283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:06.702704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:06.713567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1333/2650] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:15:10.421402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:10.433621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98916c-646d-477a-8714-eed394602de2 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:10.438694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:10.439849 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:10.440125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:10.440498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:10.442294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:10.455424 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:10.457087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:10.467450 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1318/2651] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:15:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:15:14.500536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:14.502369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:14.516040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135db3-dc5a-4819-b2da-578014c482c4 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:14.519943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:14.520605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:14.520885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:14.521158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] PATH_INFO: `/v3/users/9132e6a676054c93ba33d95d28ded4da` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:14.522286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9132e6a676054c93ba33d95d28ded4da)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:14.524162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:14.742322 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9132e6a676054c93ba33d95d28ded4da'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:14.749245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9132e6a676054c93ba33d95d28ded4da'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:14.749917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9132e6a676054c93ba33d95d28ded4da'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:14.750621 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9132e6a676054c93ba33d95d28ded4da'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:14.860091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9132e6a676054c93ba33d95d28ded4da'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:14.879666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1334/2652] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:15:14 2022] DELETE /identity/v3/users/9132e6a676054c93ba33d95d28ded4da => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:15:15.165013 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:15.178098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a5158-0a89-4f0a-99e9-c90fb2319b80 service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:15.236483 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:15.237113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:15.237391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:15.237660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] PATH_INFO: `/v3/projects/0a6caf631e5849f6867042a1f6dcf6aa` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:15.239351 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6caf631e5849f6867042a1f6dcf6aa)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:15.241285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:15.369906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6caf631e5849f6867042a1f6dcf6aa', 'request_id': 'req-bc553643-f6e4-4231-91c8-56343e6b4ef7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.465866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6caf631e5849f6867042a1f6dcf6aa', 'request_id': 'req-bc553643-f6e4-4231-91c8-56343e6b4ef7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.472014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6caf631e5849f6867042a1f6dcf6aa', 'request_id': 'req-bc553643-f6e4-4231-91c8-56343e6b4ef7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.483640 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] The token cache is being invalidate because project 0a6caf631e5849f6867042a1f6dcf6aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:15:15.483930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.484578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.486063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1319/2653] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:15:15 2022] DELETE /identity/v3/projects/0a6caf631e5849f6867042a1f6dcf6aa => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:15:15.490679 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:15.492088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:15.506162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803d2a9-f8f3-40be-9ada-b0773d014062 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:15.561397 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:15.562030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:15.562312 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:15.562587 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] PATH_INFO: `/v3/users/da283172c12f4a81b4c5c87fd138eec8` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:15.563717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da283172c12f4a81b4c5c87fd138eec8)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:15.565693 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:15.770282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da283172c12f4a81b4c5c87fd138eec8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.777263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da283172c12f4a81b4c5c87fd138eec8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.777916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da283172c12f4a81b4c5c87fd138eec8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.778536 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da283172c12f4a81b4c5c87fd138eec8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.782090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:15:15.859680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da283172c12f4a81b4c5c87fd138eec8'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:15.877672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1335/2654] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:15:15 2022] DELETE /identity/v3/users/da283172c12f4a81b4c5c87fd138eec8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:15:16.112007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:16.113485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:16.126401 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc553643-f6e4-4231-91c8-56343e6b4ef7 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:16.198601 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:16.199301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:16.199512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:16.199783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] PATH_INFO: `/v3/projects/523583963f30429a9a36115c85777964` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:16.200972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523583963f30429a9a36115c85777964)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:16.202934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:16.210592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:15:16.296577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523583963f30429a9a36115c85777964', 'request_id': 'req-169f899b-3035-4ae0-8bc5-b77acfaf6517'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:16.385795 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523583963f30429a9a36115c85777964', 'request_id': 'req-169f899b-3035-4ae0-8bc5-b77acfaf6517'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:16.392411 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523583963f30429a9a36115c85777964', 'request_id': 'req-169f899b-3035-4ae0-8bc5-b77acfaf6517'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:16.403512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] The token cache is being invalidate because project 523583963f30429a9a36115c85777964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:15:16.403845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:16.404490 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:15:16.405960 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1320/2655] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:15:16 2022] DELETE /identity/v3/projects/523583963f30429a9a36115c85777964 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:15:25.991407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:15:26.400719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:15:57.316214 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:15:57.317678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:57.376060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274db8f3-d06e-4a9d-be50-16497e327bf3 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:15:57.424552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:15:57.425160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:15:57.425438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:15:57.425710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:15:57.427470 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:15:57.489524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:15:57.490898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:15:57.501146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1336/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:01.226125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:01.227359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:01.240686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f899b-3035-4ae0-8bc5-b77acfaf6517 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:01.246219 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:01.246954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:01.247255 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:01.247554 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] PATH_INFO: `/v3/users/fb6a740de8b9417b89afb08f230aa4b7` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:01.248871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6a740de8b9417b89afb08f230aa4b7)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:01.251122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:01.415410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6a740de8b9417b89afb08f230aa4b7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:01.422451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6a740de8b9417b89afb08f230aa4b7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:01.470491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6a740de8b9417b89afb08f230aa4b7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:01.471046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6a740de8b9417b89afb08f230aa4b7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:01.483305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-585f441d-7b9d-4752-9994-521a989e07f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6a740de8b9417b89afb08f230aa4b7'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:01.489153 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1321/2657] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:01 2022] DELETE /identity/v3/users/fb6a740de8b9417b89afb08f230aa4b7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:01.747840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:01.757761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3598a4-a9be-49f2-b8d1-583637c568de service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:01.822399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:01.823028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:01.823303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:01.823575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] PATH_INFO: `/v3/projects/b40443d415fa40338b837dd445032342` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:01.827077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40443d415fa40338b837dd445032342)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:01.827077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:01.834542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:01.962971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40443d415fa40338b837dd445032342', 'request_id': 'req-ec48fe6f-78c2-4b0b-a618-03e623f22545'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:02.030724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40443d415fa40338b837dd445032342', 'request_id': 'req-ec48fe6f-78c2-4b0b-a618-03e623f22545'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:02.036481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40443d415fa40338b837dd445032342', 'request_id': 'req-ec48fe6f-78c2-4b0b-a618-03e623f22545'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:02.048250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] The token cache is being invalidate because project b40443d415fa40338b837dd445032342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:02.048596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:02.049292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ec48fe6f-78c2-4b0b-a618-03e623f22545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:02.050773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1337/2658] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:01 2022] DELETE /identity/v3/projects/b40443d415fa40338b837dd445032342 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:02.056556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.056857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.057138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.076971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:02.078175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:02.136395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1322/2659] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:02.145136 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.145421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.145694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.165591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:02.166806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:02.174462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1338/2660] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:02.179226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.188980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ae216-b80f-4b3f-b003-d7d8b62eb85f None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.193461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.194078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.194438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.194714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.195363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:16:02.195691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.197305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.208612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1323/2661] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:16:02.213068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.222298 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91412b2-34ee-46e1-96ba-b9bd0bb4193f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.227757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.228389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.228673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.228948 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.229575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.231464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.323789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1339/2662] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:02.328014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.338828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bf447-363a-4f87-868c-daf97e2ca47e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.393180 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.393771 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.394041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.394309 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.394924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.396846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.405384 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:02.470973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1324/2663] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:16:02.475097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.486223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d59b89-c54c-4a08-aa71-98f091f5d88e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.491160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.491819 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.492177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.492529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.493205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.494765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.504952 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1340/2664] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:02.513305 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.523942 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4f1d6-73ff-473b-b294-dedd11390f0c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.528681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7abd51-5797-4202-afef-f13e2308c2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.529252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7abd51-5797-4202-afef-f13e2308c2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.529531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7abd51-5797-4202-afef-f13e2308c2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.529804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7abd51-5797-4202-afef-f13e2308c2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.530457 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7abd51-5797-4202-afef-f13e2308c2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.532063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7abd51-5797-4202-afef-f13e2308c2a1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.544793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1325/2665] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:02.549066 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.558892 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6542c4-c06e-40f3-aab6-63835f444fd4 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.563469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.564112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.564422 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.564702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] PATH_INFO: `/v3/projects/f5cdb071f20945b8abedcac5bcfbad0b/users/0fc247c6885e4271bd522392133086cc/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.579425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5cdb071f20945b8abedcac5bcfbad0b, user_id=0fc247c6885e4271bd522392133086cc, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.582031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.699135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1341/2666] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:02 2022] PUT /identity/v3/projects/f5cdb071f20945b8abedcac5bcfbad0b/users/0fc247c6885e4271bd522392133086cc/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:02.710068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.710379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.710671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.738443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] MFA Rules not processed for user `0fc247c6885e4271bd522392133086cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:02.739633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:02.803748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1326/2667] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:02.808258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.868471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb99412-69eb-4c99-a323-7f580990229d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:02.872924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:02.873519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:02.873815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:02.874070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:02.874692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:02.876585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:02.974815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1342/2668] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:02.978881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:02.989659 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe0a11-3424-4ead-94fa-4279664a77aa None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:03.050221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:03.050835 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.051115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.051390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.052023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:03.054073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:03.110456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1327/2669] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:16:02 2022] POST /identity/v3/users => generated 295 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:03.114447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:03.125193 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df8f6e-4420-4905-bd31-0ef836653e27 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:03.129904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:03.130516 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.130799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.131077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.132414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:03.134132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:03.136220 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:03.145071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1343/2670] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:03.149165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:03.158989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77715-4f36-4500-bb40-751ca6230351 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:03.163917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:03.164587 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.164861 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.165131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] PATH_INFO: `/v3/projects/5ac1da1d026b4053a59addb6914dd5be/users/239a1ce3d3644ca7a94db295824cbb77/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.180509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac1da1d026b4053a59addb6914dd5be, user_id=239a1ce3d3644ca7a94db295824cbb77, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:03.183184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:03.277135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1328/2671] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:03 2022] PUT /identity/v3/projects/5ac1da1d026b4053a59addb6914dd5be/users/239a1ce3d3644ca7a94db295824cbb77/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:03.282751 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:03.293280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d32e9-0c46-4d10-8735-8f9cc84fef7a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:03.348772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d8eee-4631-42fb-8e72-bd59d90e166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:03.349398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d8eee-4631-42fb-8e72-bd59d90e166e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.349796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d8eee-4631-42fb-8e72-bd59d90e166e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.350104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d8eee-4631-42fb-8e72-bd59d90e166e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.350801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d8eee-4631-42fb-8e72-bd59d90e166e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:03.352426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d8eee-4631-42fb-8e72-bd59d90e166e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:03.362013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1344/2672] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:03.367693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:03.378078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7445d6-620e-4679-97b3-8b184b053550 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:03.383696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:03.384391 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.384686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.384948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] PATH_INFO: `/v3/projects/5ac1da1d026b4053a59addb6914dd5be/users/239a1ce3d3644ca7a94db295824cbb77/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.386874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac1da1d026b4053a59addb6914dd5be, user_id=239a1ce3d3644ca7a94db295824cbb77, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:03.389503 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:03.400147 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1329/2673] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:03 2022] PUT /identity/v3/projects/5ac1da1d026b4053a59addb6914dd5be/users/239a1ce3d3644ca7a94db295824cbb77/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:16:03.410326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.410613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.410883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.439343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] MFA Rules not processed for user `239a1ce3d3644ca7a94db295824cbb77`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:03.440577 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:03.508711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1345/2674] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:03.519372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:03.521104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:03.545523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:03.599690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72806c11-a92d-4f41-99b3-9915a4691649 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:03.604008 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:03.604639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:03.604913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:03.605182 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:03.659429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:03.660828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:03.670616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1330/2675] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:11.013615 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:11.023755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a7245-0390-4548-95fb-34a1e30a95b5 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:11.034545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:11.034545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:11.034545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:11.034545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:11.039224 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:11.040651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:11.050628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1346/2676] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:13.666713 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=48 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:17.354994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:17.357184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:17.414485 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8282d7c-b6d3-4b44-bd4d-4dcdac582b1b service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:17.418863 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:17.419468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:17.419820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:17.420090 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:17.421929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:17.502937 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:17.504499 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41f351-e740-45af-9aa9-dea69c129a01 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:17.522590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1331/2677] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:21.046412 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=70 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:27.519973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=61 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:43.745818 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:43.809296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625e32e-d639-4a91-af86-dd6a70e4b9a4 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:43.813876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:43.814537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:43.814820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:43.815111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] PATH_INFO: `/v3/users/0fc247c6885e4271bd522392133086cc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:43.816205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc247c6885e4271bd522392133086cc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:43.818227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:44.085687 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc247c6885e4271bd522392133086cc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:44.093064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc247c6885e4271bd522392133086cc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:44.094373 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc247c6885e4271bd522392133086cc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:44.094984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc247c6885e4271bd522392133086cc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:44.195730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc247c6885e4271bd522392133086cc'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:44.232451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1347/2678] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:43 2022] DELETE /identity/v3/users/0fc247c6885e4271bd522392133086cc => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:44.236996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:44.237509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:44.237674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:44.257683 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:44.258948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:44.324952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1332/2679] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:44.338071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:44.339584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:44.364639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:44.366051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:44.409725 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6acd25-f1c1-4be4-809d-43f6724dec7a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:44.413717 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:44.418441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:44.418441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:44.418441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:44.424634 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:44.425981 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74b43b-22cf-42d8-a24d-b30a42a6ca79 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:44.426115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:44.429572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:44.430178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:44.430452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:44.430719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:44.432464 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:44.435605 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1348/2680] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:16:44.445907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:44.450970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:44.459799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1333/2681] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:16:45.401281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:45.402738 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:45.416289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811edb4-0343-47ef-9cda-8f6616051b65 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:45.422402 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:45.423049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:45.423331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:45.423603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] PATH_INFO: `/v3/projects/f5cdb071f20945b8abedcac5bcfbad0b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:45.424778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5cdb071f20945b8abedcac5bcfbad0b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:45.426674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:45.445144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:45.569184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5cdb071f20945b8abedcac5bcfbad0b', 'request_id': 'req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:45.647166 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5cdb071f20945b8abedcac5bcfbad0b', 'request_id': 'req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:45.653334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5cdb071f20945b8abedcac5bcfbad0b', 'request_id': 'req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:45.665020 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] The token cache is being invalidate because project f5cdb071f20945b8abedcac5bcfbad0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:45.665572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:45.666746 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:45.668273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1349/2682] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:45 2022] DELETE /identity/v3/projects/f5cdb071f20945b8abedcac5bcfbad0b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:45.672355 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:45.682009 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cebd4d-b92f-4c94-bd86-8e1614daa909 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:45.740656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:45.741295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:45.741579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:45.741857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] PATH_INFO: `/v3/users/239a1ce3d3644ca7a94db295824cbb77` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:45.742998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239a1ce3d3644ca7a94db295824cbb77)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:45.744999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:45.958211 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:45.960125 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239a1ce3d3644ca7a94db295824cbb77'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:45.969063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239a1ce3d3644ca7a94db295824cbb77'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:46.041618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239a1ce3d3644ca7a94db295824cbb77'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:46.042208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239a1ce3d3644ca7a94db295824cbb77'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:46.056007 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-e186f240-22c3-461c-9d87-e72416256533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239a1ce3d3644ca7a94db295824cbb77'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:46.062033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1334/2683] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:45 2022] DELETE /identity/v3/users/239a1ce3d3644ca7a94db295824cbb77 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:16:46.860425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:46.862002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:46.891554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4b6c5-3a8b-49dc-9c18-4d66fda1c10c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:46.958730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:46.959927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:46.960491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:46.961008 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] PATH_INFO: `/v3/projects/5ac1da1d026b4053a59addb6914dd5be` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:46.962567 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac1da1d026b4053a59addb6914dd5be)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:46.977977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.132702 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac1da1d026b4053a59addb6914dd5be', 'request_id': 'req-35e1a75d-10a3-4a66-8e27-362128ac1d33'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:47.219420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac1da1d026b4053a59addb6914dd5be', 'request_id': 'req-35e1a75d-10a3-4a66-8e27-362128ac1d33'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:47.228472 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac1da1d026b4053a59addb6914dd5be', 'request_id': 'req-35e1a75d-10a3-4a66-8e27-362128ac1d33'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:47.242313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] The token cache is being invalidate because project 5ac1da1d026b4053a59addb6914dd5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:47.242603 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:47.243267 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-35e1a75d-10a3-4a66-8e27-362128ac1d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:47.244828 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1350/2684] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:46 2022] DELETE /identity/v3/projects/5ac1da1d026b4053a59addb6914dd5be => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:47.249812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.250323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.250513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.271924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:47.273173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:47.340972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1335/2685] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:47.349361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.350168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.350513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.386843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:47.388542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:47.397491 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1351/2686] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:16:47.401315 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:47.413004 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bb50d-611e-4df3-b586-f106ab69e581 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:47.418252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:47.418872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.419246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.419519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.420204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:16:47.420551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:47.422829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.435174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1336/2687] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:16:47.439658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:47.450139 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944db32b-3d6b-4c9e-9c6b-b3af41575ec1 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:47.455535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:47.457606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.457606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.457606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.457606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:47.459179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.586252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1352/2688] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:16:47 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:47.590434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:47.601548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c6b4a-0dc9-4bd4-ae65-868717789057 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:47.658986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:47.666296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.666296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.666296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.666296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:47.666296 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.733050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1337/2689] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:16:47 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:16:47.737290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:47.749098 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c0b79-1615-4e0f-a58a-67997dd3b05a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:47.754111 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:47.754727 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.755002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.755268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.755913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:47.757525 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.759054 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:47.769302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1353/2690] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:47.773400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:47.785016 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214e9f-1a57-4d0a-8dd4-2d350b52582a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:47.791572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f835478-ef54-4e26-8563-e7cad909ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:47.791572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f835478-ef54-4e26-8563-e7cad909ac71 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.791572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f835478-ef54-4e26-8563-e7cad909ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.791572 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f835478-ef54-4e26-8563-e7cad909ac71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.792028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f835478-ef54-4e26-8563-e7cad909ac71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:47.793559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f835478-ef54-4e26-8563-e7cad909ac71 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.805071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1338/2691] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:47.809162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:47.820680 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe41bec-6f4d-4624-824d-77c47b67c50e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:47.830822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:47.831918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.832411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.832686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] PATH_INFO: `/v3/projects/f6dc3feb2e184d5e9f94c52c1c3272e9/users/131a863e4cd043449514d2e59ab9adb9/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.850178 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dc3feb2e184d5e9f94c52c1c3272e9, user_id=131a863e4cd043449514d2e59ab9adb9, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:47.853627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:47.955550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1354/2692] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:47 2022] PUT /identity/v3/projects/f6dc3feb2e184d5e9f94c52c1c3272e9/users/131a863e4cd043449514d2e59ab9adb9/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:47.957164 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:47.957451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:47.957728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:47.989707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] MFA Rules not processed for user `131a863e4cd043449514d2e59ab9adb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:47.990998 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:48.019063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:48.075925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1339/2693] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:16:48.082755 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.168562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d6aaf-1e56-474f-8f53-5fe655f6b084 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:48.173926 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:48.174611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.174956 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.175234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.175878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:48.177833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:48.261343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1355/2694] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:16:48 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:48.265332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.283185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6305b9-c8a3-408f-a47d-ecc0086890ef None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:48.357476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:48.358087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.358364 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.358634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.359858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:48.361846 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:48.443756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1340/2695] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:16:48 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:16:48.447647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.457935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723c7b9-c6f2-426b-9725-babd926dc086 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:48.462535 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:48.463122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.463401 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.463692 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.464350 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:48.465919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:48.476411 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1356/2696] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:48.480569 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.491310 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed53d5-53ed-442a-9c27-c3435a8248ce admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:48.496005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:48.496887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.497168 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.497438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] PATH_INFO: `/v3/projects/0679def361614525abee6f7e624e1484/users/81b9327af0b0486d9a6dce2dd68a941b/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.512825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0679def361614525abee6f7e624e1484, user_id=81b9327af0b0486d9a6dce2dd68a941b, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:48.515642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:48.597218 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1341/2697] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:48 2022] PUT /identity/v3/projects/0679def361614525abee6f7e624e1484/users/81b9327af0b0486d9a6dce2dd68a941b/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:48.601151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.611334 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cee7-f4ea-41fd-bed7-282c8c65a75e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:48.668694 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e9db0-1f45-4471-9f99-8012dc870bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:48.669218 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e9db0-1f45-4471-9f99-8012dc870bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.669494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e9db0-1f45-4471-9f99-8012dc870bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.669766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e9db0-1f45-4471-9f99-8012dc870bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.670405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e9db0-1f45-4471-9f99-8012dc870bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:48.672071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e9db0-1f45-4471-9f99-8012dc870bb2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:48.682002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1357/2698] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:48.686249 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.697073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e19b6-ba4d-458f-8af8-54974be913b2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:48.701699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:48.702357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.702692 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.703570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] PATH_INFO: `/v3/projects/0679def361614525abee6f7e624e1484/users/81b9327af0b0486d9a6dce2dd68a941b/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.705900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0679def361614525abee6f7e624e1484, user_id=81b9327af0b0486d9a6dce2dd68a941b, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:48.708565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:48.719766 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1342/2699] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:48 2022] PUT /identity/v3/projects/0679def361614525abee6f7e624e1484/users/81b9327af0b0486d9a6dce2dd68a941b/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:48.730959 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:48.731245 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:48.731524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:48.761494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] MFA Rules not processed for user `81b9327af0b0486d9a6dce2dd68a941b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:48.762741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:48.793741 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:48.833059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1358/2700] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:48.845089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:48.847178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:49.038758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242e78-5729-4a5b-bfa5-99b4f040d021 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:49.042584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b360857-50a2-4440-911c-406874e62e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:49.043195 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360857-50a2-4440-911c-406874e62e5f service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:49.043470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360857-50a2-4440-911c-406874e62e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:49.043737 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360857-50a2-4440-911c-406874e62e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:49.055612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b360857-50a2-4440-911c-406874e62e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:49.057385 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b360857-50a2-4440-911c-406874e62e5f service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:49.068006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1343/2701] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:49.099436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:49.165087 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375e16d-0e76-4ddf-aef3-2f3e13d19d42 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:49.169609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:49.170244 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:49.170523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:49.170792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] PATH_INFO: `/v3/users/131a863e4cd043449514d2e59ab9adb9` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:49.171935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=131a863e4cd043449514d2e59ab9adb9)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:49.173858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:49.348163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131a863e4cd043449514d2e59ab9adb9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:49.355296 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131a863e4cd043449514d2e59ab9adb9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:49.356027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131a863e4cd043449514d2e59ab9adb9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:49.356704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131a863e4cd043449514d2e59ab9adb9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:49.470524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131a863e4cd043449514d2e59ab9adb9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:49.500371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1359/2702] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:49 2022] DELETE /identity/v3/users/131a863e4cd043449514d2e59ab9adb9 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:49.506185 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:49.506669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:49.506850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:49.532638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:49.533834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:49.543613 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:49.608511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1344/2703] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:49.616042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:49.617658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:49.679026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:49.686126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce8d7f-c6fc-47fc-aa23-08fbb134d596 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:49.690520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:49.691147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:49.691518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:49.691810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:49.705275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:49.706732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:49.716774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1360/2704] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:50.238463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:50.239983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:50.254153 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b21883-27c0-4944-8f47-e9e08c741ff6 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:50.261130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:50.261813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:50.262087 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:50.262357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] PATH_INFO: `/v3/projects/f6dc3feb2e184d5e9f94c52c1c3272e9` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:50.263507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6dc3feb2e184d5e9f94c52c1c3272e9)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:50.265529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:50.411542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6dc3feb2e184d5e9f94c52c1c3272e9', 'request_id': 'req-f1c616f7-10f2-4412-8536-33abc98fca9e'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.573207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6dc3feb2e184d5e9f94c52c1c3272e9', 'request_id': 'req-f1c616f7-10f2-4412-8536-33abc98fca9e'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.581373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6dc3feb2e184d5e9f94c52c1c3272e9', 'request_id': 'req-f1c616f7-10f2-4412-8536-33abc98fca9e'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.595757 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] The token cache is being invalidate because project f6dc3feb2e184d5e9f94c52c1c3272e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:50.596099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.596804 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.598585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1345/2705] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:50 2022] DELETE /identity/v3/projects/f6dc3feb2e184d5e9f94c52c1c3272e9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:50.602646 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:50.615841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2b85c-f109-45f2-91c5-c9bb1fd0cf86 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:50.685062 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:50.685699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:50.686421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:50.686421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] PATH_INFO: `/v3/users/81b9327af0b0486d9a6dce2dd68a941b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:50.687534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b9327af0b0486d9a6dce2dd68a941b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:50.689589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:50.963368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b9327af0b0486d9a6dce2dd68a941b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.971112 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b9327af0b0486d9a6dce2dd68a941b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.971854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b9327af0b0486d9a6dce2dd68a941b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:50.972647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b9327af0b0486d9a6dce2dd68a941b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:51.093104 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-28d5702c-3836-4709-bd37-f5e328896939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b9327af0b0486d9a6dce2dd68a941b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:51.117300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1361/2706] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:50 2022] DELETE /identity/v3/users/81b9327af0b0486d9a6dce2dd68a941b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:51.592552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:51.594935 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:51.631342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c616f7-10f2-4412-8536-33abc98fca9e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:51.697800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:51.698476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:51.698762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:51.699034 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] PATH_INFO: `/v3/projects/0679def361614525abee6f7e624e1484` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:51.700392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0679def361614525abee6f7e624e1484)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:51.702295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:51.886262 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:51.895189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0679def361614525abee6f7e624e1484', 'request_id': 'req-f78c6511-9f3e-4f03-833b-e133b633dd33'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:51.961948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0679def361614525abee6f7e624e1484', 'request_id': 'req-f78c6511-9f3e-4f03-833b-e133b633dd33'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:51.969285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0679def361614525abee6f7e624e1484', 'request_id': 'req-f78c6511-9f3e-4f03-833b-e133b633dd33'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:51.996668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] The token cache is being invalidate because project 0679def361614525abee6f7e624e1484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:51.996894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:51.997929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-f78c6511-9f3e-4f03-833b-e133b633dd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:52.001133 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1346/2707] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:51 2022] DELETE /identity/v3/projects/0679def361614525abee6f7e624e1484 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:52.006399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.006897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.007078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.030462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:52.032353 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:52.110532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1362/2708] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:16:52.118779 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.119263 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.119545 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.139668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:52.140922 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:52.149065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1347/2709] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:52.153519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.163562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac6997-890a-4d89-a7d9-49203780d88f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.168380 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.168992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.169275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.169545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.170198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:16:52.170521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.173230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.179082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:52.190728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1363/2710] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:16:52.190884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.192172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.192495 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:52.203988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17764009-a51c-4424-ae7e-860a1495d279 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.210104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd76ff-53c8-42bb-8cef-b9485f38df89 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.210216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.210216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.210216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.210216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.210794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.211342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.211970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.212247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.212550 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] PATH_INFO: `/v3/users/14bd04e951254ec5bc29999bceeea214` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.212723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.213847 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14bd04e951254ec5bc29999bceeea214)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.215746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.311851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1364/2711] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:16:52 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:52.316560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.381081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a75ba-e11b-49da-b5b8-61df4d3ced14 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.385841 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.386469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.386773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.387028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.387669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.389593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.429173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14bd04e951254ec5bc29999bceeea214'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:52.436399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14bd04e951254ec5bc29999bceeea214'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:52.502071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1365/2712] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:16:52 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:52.503518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.513237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee747-ee46-4450-b6a4-46cc23186e6b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.518164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.518757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.519060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.519337 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.519983 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.521285 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14bd04e951254ec5bc29999bceeea214'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:52.521619 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.522116 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14bd04e951254ec5bc29999bceeea214'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:52.537170 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14bd04e951254ec5bc29999bceeea214'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:52.541602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1366/2713] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:16:52.542502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.544212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:52.548476 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1348/2714] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:52 2022] DELETE /identity/v3/users/14bd04e951254ec5bc29999bceeea214 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:16:52.558389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11575e3-3442-46d6-894f-abc5222b518d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.618311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a512fc1-a414-462b-b698-3658a307628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.618929 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a512fc1-a414-462b-b698-3658a307628e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.619210 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a512fc1-a414-462b-b698-3658a307628e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.619484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a512fc1-a414-462b-b698-3658a307628e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.620279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a512fc1-a414-462b-b698-3658a307628e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.621913 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a512fc1-a414-462b-b698-3658a307628e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.631975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1367/2715] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:52.636028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.646638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6047-2f71-4d60-8e11-b615acc65e40 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.652390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.653410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.654334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.654796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] PATH_INFO: `/v3/projects/e07ff6d99d7747b19dc9b1aa840003cb/users/fefe5312d5b24549aa53d0a1ce12787b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.672544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07ff6d99d7747b19dc9b1aa840003cb, user_id=fefe5312d5b24549aa53d0a1ce12787b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.675228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.785470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1349/2716] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:52 2022] PUT /identity/v3/projects/e07ff6d99d7747b19dc9b1aa840003cb/users/fefe5312d5b24549aa53d0a1ce12787b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:52.796069 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.796458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.796743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.820777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.822892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:52.825034 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] MFA Rules not processed for user `fefe5312d5b24549aa53d0a1ce12787b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:52.826252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:52.894059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1368/2717] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:52.897939 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332b40-7620-47fb-843d-1cb88972589c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.898386 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:52.902669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.903386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.903658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.903928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] PATH_INFO: `/v3/projects/3ca8eec30c0b4a659620b6f45bb94db9` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.905095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca8eec30c0b4a659620b6f45bb94db9)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.907541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.909670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9faed-39a0-45a9-8def-b78e96565839 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:52.914625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:52.915263 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:52.915573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:52.915852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:52.916512 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:52.918419 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:52.962228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:53.016932 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca8eec30c0b4a659620b6f45bb94db9', 'request_id': 'req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.030964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1369/2718] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:16:52 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:16:53.032346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.096262 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35745ae1-d645-4c97-bb46-9098898ffc09 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.100969 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.101594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.101878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.102150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.102877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.105094 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.138606 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca8eec30c0b4a659620b6f45bb94db9', 'request_id': 'req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.144620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca8eec30c0b4a659620b6f45bb94db9', 'request_id': 'req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.155533 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] The token cache is being invalidate because project 3ca8eec30c0b4a659620b6f45bb94db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:53.156374 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.157038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.158529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1350/2719] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:52 2022] DELETE /identity/v3/projects/3ca8eec30c0b4a659620b6f45bb94db9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:16:53.162820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.164135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:53.232001 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0ae26-83d2-4189-9ac9-d9648ef0bf31 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.237421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.238196 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.238470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.238741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] PATH_INFO: `/v3/users/8cbcb286b19741a1a41a5b86b697a645` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.249890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1370/2720] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:16:53 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:16:53.253025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.254783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:53.255297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbcb286b19741a1a41a5b86b697a645)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.257325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.267217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0308e-8570-4ca1-8a53-e4053251f936 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.272228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.272878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.273158 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.273423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.274454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.275808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.287286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1371/2721] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:16:53.292344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.303830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affc0b6-b734-475c-8841-ef60e15717d9 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.309698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.316988 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.317461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.317773 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] PATH_INFO: `/v3/projects/910d02ce1eac4b8d923abada041da6de/users/83d0b161b42b4b1fa4ccf88ff4481582/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.333930 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910d02ce1eac4b8d923abada041da6de, user_id=83d0b161b42b4b1fa4ccf88ff4481582, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.336600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.465916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:53.478632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1372/2722] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:53 2022] PUT /identity/v3/projects/910d02ce1eac4b8d923abada041da6de/users/83d0b161b42b4b1fa4ccf88ff4481582/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:16:53.483091 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.492879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbcb286b19741a1a41a5b86b697a645'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.501203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbcb286b19741a1a41a5b86b697a645'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.547918 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9709f-5341-4a28-8bda-7b63fc827a61 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.552941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.553559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.553923 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.554257 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.554911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.556584 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.567174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1373/2723] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:16:53.571616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.579893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbcb286b19741a1a41a5b86b697a645'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.580556 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbcb286b19741a1a41a5b86b697a645'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.585117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0560c-b9f7-496a-801d-7dee669c0f9e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.591359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.591636 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.591916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.592192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] PATH_INFO: `/v3/projects/910d02ce1eac4b8d923abada041da6de/users/83d0b161b42b4b1fa4ccf88ff4481582/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.594138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-5d79a340-f45c-4ffc-aa4d-026f63f42ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbcb286b19741a1a41a5b86b697a645'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:53.595013 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910d02ce1eac4b8d923abada041da6de, user_id=83d0b161b42b4b1fa4ccf88ff4481582, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.597662 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.600718 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1351/2724] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:16:53 2022] DELETE /identity/v3/users/8cbcb286b19741a1a41a5b86b697a645 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:16:53.612860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1374/2725] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:16:53 2022] PUT /identity/v3/projects/910d02ce1eac4b8d923abada041da6de/users/83d0b161b42b4b1fa4ccf88ff4481582/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:16:53.629409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.629690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.629974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.660833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] MFA Rules not processed for user `83d0b161b42b4b1fa4ccf88ff4481582`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:16:53.662127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:53.732564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1352/2726] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:16:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:16:53.744691 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.746369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:53.812572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762089-f43c-4729-bac1-3ad23f476640 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:53.816891 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:53.817507 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:53.817782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:53.818052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:53.828039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:53.829670 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:53.839377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1375/2727] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:16:53.906375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:53.907943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:53.916275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:53.979101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:16:54.002121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828c72-2739-4ca9-9363-13d9e9a3eb67 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:54.006772 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:54.007400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:54.007680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:54.007953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] PATH_INFO: `/v3/projects/d43808494dc44e9fbcc5b75e60c54b42` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:54.019559 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d43808494dc44e9fbcc5b75e60c54b42)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:54.021517 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:54.136697 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:16:54.145849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30fc3e-3340-49ae-b676-f17863ee6a8d service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:16:54.149523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:16:54.150113 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:16:54.150390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:16:54.150659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:16:54.152352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:16:54.173507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd43808494dc44e9fbcc5b75e60c54b42', 'request_id': 'req-1d8562fb-844a-4a71-912d-c45455f3ffd1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:54.211556 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:16:54.212932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:16:54.221776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1376/2728] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:16:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:16:54.258359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd43808494dc44e9fbcc5b75e60c54b42', 'request_id': 'req-1d8562fb-844a-4a71-912d-c45455f3ffd1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:54.265905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd43808494dc44e9fbcc5b75e60c54b42', 'request_id': 'req-1d8562fb-844a-4a71-912d-c45455f3ffd1'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:54.277522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] The token cache is being invalidate because project d43808494dc44e9fbcc5b75e60c54b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:16:54.277812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:54.278453 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-1d8562fb-844a-4a71-912d-c45455f3ffd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:16:54.280051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1353/2729] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:16:53 2022] DELETE /identity/v3/projects/d43808494dc44e9fbcc5b75e60c54b42 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:00.530206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:00.532285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:00.545817 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5d911-01a3-4818-8f72-dd5238f92872 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:00.601288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:00.602001 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:00.602277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:00.602550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] PATH_INFO: `/v3/users/fefe5312d5b24549aa53d0a1ce12787b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:00.603666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fefe5312d5b24549aa53d0a1ce12787b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:00.605639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:00.790223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:00.798165 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fefe5312d5b24549aa53d0a1ce12787b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:00.805177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fefe5312d5b24549aa53d0a1ce12787b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:00.805873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fefe5312d5b24549aa53d0a1ce12787b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:00.806464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fefe5312d5b24549aa53d0a1ce12787b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:00.863976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fefe5312d5b24549aa53d0a1ce12787b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:00.883202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1377/2730] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:17:00 2022] DELETE /identity/v3/users/fefe5312d5b24549aa53d0a1ce12787b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:00.887686 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:00.888175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:00.888334 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:00.907514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:00.908762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:00.917469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:00.969999 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1354/2731] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:00.986330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:00.987859 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:01.045216 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cffeae-c457-44fb-be0c-431038ab9a64 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:01.049485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:01.050089 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:01.050368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:01.050641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:01.059538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:01.061010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:01.071065 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1378/2732] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:17:01.457833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:01.459522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:01.475142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c65b25-6103-4e47-ba49-dd3c2c9f9f0e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:01.480719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:01.481541 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:01.481887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:01.482102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] PATH_INFO: `/v3/projects/e07ff6d99d7747b19dc9b1aa840003cb` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:01.483214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07ff6d99d7747b19dc9b1aa840003cb)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:01.485865 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:01.572592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07ff6d99d7747b19dc9b1aa840003cb', 'request_id': 'req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.630392 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07ff6d99d7747b19dc9b1aa840003cb', 'request_id': 'req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.636701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07ff6d99d7747b19dc9b1aa840003cb', 'request_id': 'req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.648265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] The token cache is being invalidate because project e07ff6d99d7747b19dc9b1aa840003cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:17:01.648589 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.649220 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.650749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1355/2733] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:17:01 2022] DELETE /identity/v3/projects/e07ff6d99d7747b19dc9b1aa840003cb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:01.658097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:01.667800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f3a5e-7b76-4e61-bced-908561705de8 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:01.728541 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:01.729173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:01.729456 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:01.729730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] PATH_INFO: `/v3/users/83d0b161b42b4b1fa4ccf88ff4481582` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:01.730885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d0b161b42b4b1fa4ccf88ff4481582)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:01.732821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:01.871781 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d0b161b42b4b1fa4ccf88ff4481582'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.878834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d0b161b42b4b1fa4ccf88ff4481582'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.879502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d0b161b42b4b1fa4ccf88ff4481582'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.880114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d0b161b42b4b1fa4ccf88ff4481582'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.882632 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:01.932172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-89be3f0d-611c-466c-a01d-c3b9fbc65471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d0b161b42b4b1fa4ccf88ff4481582'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:01.950114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1379/2734] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:17:01 2022] DELETE /identity/v3/users/83d0b161b42b4b1fa4ccf88ff4481582 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:02.392422 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:02.394614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:02.407900 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4a67f-8cca-48b1-b4cb-ad7009a5017a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:02.469956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:02.470586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:02.470860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:02.471131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] PATH_INFO: `/v3/projects/910d02ce1eac4b8d923abada041da6de` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:02.472280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=910d02ce1eac4b8d923abada041da6de)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:02.475933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:02.588582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:02.595423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910d02ce1eac4b8d923abada041da6de', 'request_id': 'req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:02.715621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910d02ce1eac4b8d923abada041da6de', 'request_id': 'req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:02.721699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910d02ce1eac4b8d923abada041da6de', 'request_id': 'req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:02.733858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] The token cache is being invalidate because project 910d02ce1eac4b8d923abada041da6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:17:02.733858 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:02.734208 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-aa00e2db-bf04-46ae-939b-ba3b9d799b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:02.735704 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1356/2735] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:17:02 2022] DELETE /identity/v3/projects/910d02ce1eac4b8d923abada041da6de => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:02.740658 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:02.741130 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:02.741230 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:02.764790 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:02.766153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:02.828775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1380/2736] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:02.836591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:02.836883 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:02.837158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:02.858047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:02.860041 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:02.867827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1357/2737] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:02.871911 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:02.881813 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e44885-2c56-427f-a370-2570f95b04ea None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:02.886400 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:02.887002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:02.887286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:02.887561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:02.888215 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:17:02.888554 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:02.890252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:02.901666 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1381/2738] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:17:02.906540 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:02.916508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6061fa-15ac-4f31-b25d-ed647daaf31e None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:02.921970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:02.922734 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:02.923011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:02.923277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:02.923902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:02.925807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:02.997190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1358/2739] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:17:02 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:17:03.002198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:03.012303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dbccc-8f0c-48a8-a5f2-cbe2b0ea50ec admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:03.068474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:03.069078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:03.069360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:03.069635 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:03.070274 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:03.072176 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:03.144795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1382/2740] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:17:03 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:17:03.148925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:03.159765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b406f-4336-40a6-ba36-96c07639e567 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:03.164504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:03.165091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:03.165388 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:03.165633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:03.166269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:03.167836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:03.178622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1359/2741] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:17:03.182488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:03.193351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91139b2-1a25-4275-a56d-62be7e02528e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:03.198934 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e2a61-e014-49d8-8282-37fbc96cbecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:03.199613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e2a61-e014-49d8-8282-37fbc96cbecc admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:03.200088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e2a61-e014-49d8-8282-37fbc96cbecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:03.200407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e2a61-e014-49d8-8282-37fbc96cbecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:03.201090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e2a61-e014-49d8-8282-37fbc96cbecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:03.203293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e2a61-e014-49d8-8282-37fbc96cbecc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:03.214471 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1383/2742] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:17:03.219362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:03.229430 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc51d13-037d-4a1b-beeb-9a1467fca682 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:03.233967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:03.234633 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:03.234890 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:03.239628 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] PATH_INFO: `/v3/projects/2811244dd42c4100a0a04c2a3066d080/users/f4622d0ae72e4f5ebcc62a75747662f3/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:03.254749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2811244dd42c4100a0a04c2a3066d080, user_id=f4622d0ae72e4f5ebcc62a75747662f3, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:03.257423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:03.336079 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1360/2743] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:17:03 2022] PUT /identity/v3/projects/2811244dd42c4100a0a04c2a3066d080/users/f4622d0ae72e4f5ebcc62a75747662f3/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:03.349315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:03.349669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:03.350010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:03.377521 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] MFA Rules not processed for user `f4622d0ae72e4f5ebcc62a75747662f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:03.378730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:03.409250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:03.442873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1384/2744] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:03.454970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:03.456719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:03.535823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41611314-ef7b-453b-8c4f-efabecb2dd57 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:03.539814 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a29e0-d7d2-4b9b-be45-9626183e930b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:03.540544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a29e0-d7d2-4b9b-be45-9626183e930b service glance] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:03.540884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a29e0-d7d2-4b9b-be45-9626183e930b service glance] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:03.541202 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a29e0-d7d2-4b9b-be45-9626183e930b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:03.551067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a29e0-d7d2-4b9b-be45-9626183e930b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:03.552738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a29e0-d7d2-4b9b-be45-9626183e930b service glance] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:03.561741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1361/2745] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:17:13.475133 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=13 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:13.557562 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=72,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:29.162028 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:29.223046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620af6-1295-4648-bdd4-bf548311381e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:29.227674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:29.228288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:29.228602 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:29.228877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] PATH_INFO: `/v3/users/f4622d0ae72e4f5ebcc62a75747662f3` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:29.229992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4622d0ae72e4f5ebcc62a75747662f3)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:29.231885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:29.519010 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4622d0ae72e4f5ebcc62a75747662f3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:29.527009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4622d0ae72e4f5ebcc62a75747662f3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:29.527772 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4622d0ae72e4f5ebcc62a75747662f3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:29.528499 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4622d0ae72e4f5ebcc62a75747662f3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:29.646317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4622d0ae72e4f5ebcc62a75747662f3'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:29.664951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1385/2746] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:17:29 2022] DELETE /identity/v3/users/f4622d0ae72e4f5ebcc62a75747662f3 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:29.669491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:29.669985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:29.670127 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:29.688669 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:29.691666 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:29.748329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1362/2747] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:29.758301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:29.759765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:29.818123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c0670-f703-476a-8d3f-b3d2e3c68474 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:29.821566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4082a-91b9-4c0d-a8f7-67244962936e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:29.822167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4082a-91b9-4c0d-a8f7-67244962936e service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:29.822444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4082a-91b9-4c0d-a8f7-67244962936e service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:29.822716 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4082a-91b9-4c0d-a8f7-67244962936e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:29.832153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4082a-91b9-4c0d-a8f7-67244962936e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:29.833623 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4082a-91b9-4c0d-a8f7-67244962936e service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:29.842852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1386/2748] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:17:30.226074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:30.226320 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:30.248092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a3d8b-23b5-4e15-b398-256151747453 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:30.253694 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:30.254344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:30.254618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:30.254889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] PATH_INFO: `/v3/projects/2811244dd42c4100a0a04c2a3066d080` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:30.256092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2811244dd42c4100a0a04c2a3066d080)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:30.257941 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:30.412429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2811244dd42c4100a0a04c2a3066d080', 'request_id': 'req-b879f39b-08d2-4d95-89b1-5035933b6966'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:30.514787 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2811244dd42c4100a0a04c2a3066d080', 'request_id': 'req-b879f39b-08d2-4d95-89b1-5035933b6966'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:30.520745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2811244dd42c4100a0a04c2a3066d080', 'request_id': 'req-b879f39b-08d2-4d95-89b1-5035933b6966'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:30.531857 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] The token cache is being invalidate because project 2811244dd42c4100a0a04c2a3066d080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:17:30.532146 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:30.532809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b879f39b-08d2-4d95-89b1-5035933b6966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:30.534288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1363/2749] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:17:30 2022] DELETE /identity/v3/projects/2811244dd42c4100a0a04c2a3066d080 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:30.539463 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:30.539921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:30.540031 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:30.559778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:30.561077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:30.578374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:30.619880 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1387/2750] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:30.628092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:30.628417 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:30.628697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:30.648092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:30.649322 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:30.657955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1364/2751] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:30.662314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:30.672718 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297ebb1-c480-461c-8b76-0a92475ee355 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:30.677238 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:30.677834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:30.678114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:30.678392 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:30.679041 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:17:30.679343 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:30.680951 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:30.692303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1388/2752] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:17:30.696513 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:30.705894 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaf14c-ab5e-4a23-a2a9-13ac7a809b13 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:30.712696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:30.713306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:30.713580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:30.713845 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:30.714497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:30.719478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:30.803685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1365/2753] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:17:30 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:17:30.808036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:30.818159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c5d66-1069-4fcb-b434-63a03d211159 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:30.874236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:30.874844 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:30.875117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:30.875387 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:30.876037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:30.877960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:31.019770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1389/2754] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:17:30 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:17:31.023911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:31.034872 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0327d-40e3-4cbd-a330-a64fe1f4a30a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:31.040777 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:31.041395 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:31.041672 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:31.041944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:31.042597 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:31.044191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:31.058642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1366/2755] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:17:31.062829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:31.075488 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b8160-7ebe-4aa4-abc3-9104d1167fa9 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:31.080743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a6419-4569-4806-aa0d-a39602ac8d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:31.081344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a6419-4569-4806-aa0d-a39602ac8d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:31.081624 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a6419-4569-4806-aa0d-a39602ac8d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:31.081897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a6419-4569-4806-aa0d-a39602ac8d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:31.082537 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a6419-4569-4806-aa0d-a39602ac8d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:31.084204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a6419-4569-4806-aa0d-a39602ac8d1c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:31.096650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1390/2756] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:17:31.100765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:31.110542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11ffcd-6127-4f25-ba8a-d260351c5744 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:31.114978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:31.115612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:31.115891 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:31.116176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] PATH_INFO: `/v3/projects/68000bd4600346618723066a685db961/users/d007d3e48604426aaad8b2f45440e8f5/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:31.130083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68000bd4600346618723066a685db961, user_id=d007d3e48604426aaad8b2f45440e8f5, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:31.132745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:31.224178 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1367/2757] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:17:31 2022] PUT /identity/v3/projects/68000bd4600346618723066a685db961/users/d007d3e48604426aaad8b2f45440e8f5/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:17:31.234769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:31.235002 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:31.235285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:31.265421 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] MFA Rules not processed for user `d007d3e48604426aaad8b2f45440e8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:31.266678 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:31.331049 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1391/2758] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:17:31.342542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:31.344590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:31.365973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:31.401194 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32384e37-aff7-4544-979f-fc1b0b69b8e8 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:31.404801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0805d-a840-4f24-bbd0-8188e16f4be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:31.405398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0805d-a840-4f24-bbd0-8188e16f4be7 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:31.405699 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0805d-a840-4f24-bbd0-8188e16f4be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:31.406015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0805d-a840-4f24-bbd0-8188e16f4be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:31.415070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0805d-a840-4f24-bbd0-8188e16f4be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:31.416515 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0805d-a840-4f24-bbd0-8188e16f4be7 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:31.425163 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1368/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:17:36.727997 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:36.747659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:36.792607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b80126-f98d-4ce4-a400-8d9b6b562c99 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:36.797852 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:36.798480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:36.798758 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:36.799029 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] PATH_INFO: `/v3/users/d007d3e48604426aaad8b2f45440e8f5` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:36.800280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d007d3e48604426aaad8b2f45440e8f5)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:36.802820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:36.992300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd007d3e48604426aaad8b2f45440e8f5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:36.999290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd007d3e48604426aaad8b2f45440e8f5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:36.999966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd007d3e48604426aaad8b2f45440e8f5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:37.000873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd007d3e48604426aaad8b2f45440e8f5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:37.064816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd007d3e48604426aaad8b2f45440e8f5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:37.085170 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1392/2760] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:17:36 2022] DELETE /identity/v3/users/d007d3e48604426aaad8b2f45440e8f5 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:37.091342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:37.091855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:37.091975 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:37.119358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:37.120698 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:37.185439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1369/2761] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:17:37.198223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:37.199654 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:37.280162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422ada6-c192-4b11-9633-8e8af546c050 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:37.284486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ca24f-913d-4990-be78-d8883c087377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:37.285097 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ca24f-913d-4990-be78-d8883c087377 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:37.285399 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ca24f-913d-4990-be78-d8883c087377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:37.285723 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ca24f-913d-4990-be78-d8883c087377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:37.294994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ca24f-913d-4990-be78-d8883c087377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:37.296436 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ca24f-913d-4990-be78-d8883c087377 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:37.305916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1393/2762] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:17:37.815000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:37.816693 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:37.829197 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509cf39-afcb-4806-864c-66e4c7f48de2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:37.833764 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:37.834400 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:37.834719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:37.834990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] PATH_INFO: `/v3/projects/68000bd4600346618723066a685db961` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:37.837651 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68000bd4600346618723066a685db961)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:37.839614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:37.972225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68000bd4600346618723066a685db961', 'request_id': 'req-a302fbe6-5746-4989-ab31-b22ca53934f2'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:38.069532 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68000bd4600346618723066a685db961', 'request_id': 'req-a302fbe6-5746-4989-ab31-b22ca53934f2'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:38.075618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68000bd4600346618723066a685db961', 'request_id': 'req-a302fbe6-5746-4989-ab31-b22ca53934f2'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:38.086943 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] The token cache is being invalidate because project 68000bd4600346618723066a685db961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:17:38.087231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:38.087871 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-a302fbe6-5746-4989-ab31-b22ca53934f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:17:38.089407 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1370/2763] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:17:37 2022] DELETE /identity/v3/projects/68000bd4600346618723066a685db961 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:38.111719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.112227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.112371 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.131752 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:38.139672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:38.167160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:38.192798 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1394/2764] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:38.200632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.201135 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.201248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.221130 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:38.222328 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:38.230123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1371/2765] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:38.235276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.245853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14128e-9575-4b73-917f-a14cd15e63db None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.250553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.254517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.254517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.254517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.254517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:17:38.254517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:38.254517 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:38.265823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1395/2766] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:17:38.270267 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.279420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae07a-d68b-4623-aaa0-8f930e07fdb1 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.284950 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.285544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.285823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.286096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.287014 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:38.289225 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:38.415028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1372/2767] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:17:38 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:17:38.419204 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.429464 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127bef6-93d6-45c7-b3aa-bb6ee2781e32 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.484336 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.484960 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.485239 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.485510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.486141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:38.488021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:38.618088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1396/2768] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:17:38 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:17:38.622257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.673507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84056a0-714e-491f-9ba5-f9714832e619 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.673507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.673507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.673507 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.673936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.673936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:38.673936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:38.673936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1373/2769] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:17:38.675520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.685674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766de33b-e47e-41cf-80da-770251462dce admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.690778 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed319694-93a0-4f06-97ca-20f144ee168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.691377 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed319694-93a0-4f06-97ca-20f144ee168b admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.691653 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed319694-93a0-4f06-97ca-20f144ee168b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.691922 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed319694-93a0-4f06-97ca-20f144ee168b admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.692596 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed319694-93a0-4f06-97ca-20f144ee168b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:38.694153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed319694-93a0-4f06-97ca-20f144ee168b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:38.705760 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1397/2770] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:17:38.709862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.721119 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630cb2f-855b-4670-8695-256509ef1b62 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.725776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.726431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.726711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.726985 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] PATH_INFO: `/v3/projects/a2d986195f844099b8616c92bc2162b4/users/9d20630980d642fa90636ff4c2d67075/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.729287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:38.742167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d986195f844099b8616c92bc2162b4, user_id=9d20630980d642fa90636ff4c2d67075, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:38.747885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:38.830200 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1374/2771] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:17:38 2022] PUT /identity/v3/projects/a2d986195f844099b8616c92bc2162b4/users/9d20630980d642fa90636ff4c2d67075/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:38.835228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1501-8b6f-4f06-b4f5-dd951013780c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.835519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1501-8b6f-4f06-b4f5-dd951013780c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.835789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1501-8b6f-4f06-b4f5-dd951013780c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:38.858628 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-25fe1501-8b6f-4f06-b4f5-dd951013780c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:38.860732 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-25fe1501-8b6f-4f06-b4f5-dd951013780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:38.921954 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1398/2772] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:38.931402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:38.933137 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:38.989921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3285875-f0d2-4d9e-9fae-65f681c21fac admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:38.993722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:38.994339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:38.994668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:38.995071 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:39.005143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:39.006566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:39.016511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1375/2773] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:17:44.881964 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:44.882514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:44.882616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:44.912466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] MFA Rules not processed for user `9d20630980d642fa90636ff4c2d67075`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:44.913857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:44.947627 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:44.978437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1399/2774] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:44.982928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:44.992624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded984ba-18fe-4046-89e6-de40c25f78ab service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:44.997167 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:44.997759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:44.998078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:44.998287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:44.999015 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:45.001389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:45.141409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1376/2775] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:17:44 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:17:45.145673 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:45.156396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8534f7-a61d-4fe0-99bc-8bdd4a5f6b90 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:45.213830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:45.214447 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:45.215117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:45.215117 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:45.215613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:45.217661 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:45.333462 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1400/2776] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:17:45 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:17:45.345897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:45.349047 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429c57-7567-45eb-9f9c-a37f56a0369c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:45.354373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:45.354812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:45.355092 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:45.355358 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:45.356005 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:45.357635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:45.370120 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1377/2777] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:17:45.375361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:45.388560 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1183b-c05c-4668-937c-4dea0c7a969a admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:45.393949 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:45.394645 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:45.394921 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:45.395305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] PATH_INFO: `/v3/projects/3bbf1e4db6d445519c083ac433d6bd1d/users/796f0da26d074d3f98e90da25c06f50d/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:45.411174 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbf1e4db6d445519c083ac433d6bd1d, user_id=796f0da26d074d3f98e90da25c06f50d, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:45.414153 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:45.539175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1401/2778] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:17:45 2022] PUT /identity/v3/projects/3bbf1e4db6d445519c083ac433d6bd1d/users/796f0da26d074d3f98e90da25c06f50d/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:17:45.543344 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:45.554505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e25d2-918e-4127-a9df-2164962a8cb9 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:45.614668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0ea4e-c5ac-4ce7-b40b-f3c8f7aa4d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:45.614668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0ea4e-c5ac-4ce7-b40b-f3c8f7aa4d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:45.614668 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0ea4e-c5ac-4ce7-b40b-f3c8f7aa4d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:45.615104 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0ea4e-c5ac-4ce7-b40b-f3c8f7aa4d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:45.616615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0ea4e-c5ac-4ce7-b40b-f3c8f7aa4d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:45.617299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0ea4e-c5ac-4ce7-b40b-f3c8f7aa4d5b admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:45.631705 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1378/2779] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:17:45.635820 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:45.646015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37c108-e235-49ed-91f2-fa4c51891b70 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:45.650643 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:45.651302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:45.651581 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:45.651854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] PATH_INFO: `/v3/projects/3bbf1e4db6d445519c083ac433d6bd1d/users/796f0da26d074d3f98e90da25c06f50d/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:45.653802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbf1e4db6d445519c083ac433d6bd1d, user_id=796f0da26d074d3f98e90da25c06f50d, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:45.657290 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:45.668103 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1402/2780] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:17:45 2022] PUT /identity/v3/projects/3bbf1e4db6d445519c083ac433d6bd1d/users/796f0da26d074d3f98e90da25c06f50d/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:17:51.490254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3fbb1-1375-4c03-89c4-6f4802cf595d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:51.490807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3fbb1-1375-4c03-89c4-6f4802cf595d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:51.490807 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3fbb1-1375-4c03-89c4-6f4802cf595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:51.534852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-e5c3fbb1-1375-4c03-89c4-6f4802cf595d None None] MFA Rules not processed for user `796f0da26d074d3f98e90da25c06f50d`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:51.536657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e5c3fbb1-1375-4c03-89c4-6f4802cf595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:51.582142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:51.613246 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1379/2781] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:51.625458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:51.627315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:51.703698 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d142a25-1b15-496b-b6b1-28868de3a496 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:51.708145 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbb0ec082bdb45b0bb2c59cbad440d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:51.708821 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:51.709101 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:51.709374 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:51.756485 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:17:51.772671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:51.774107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:51.783420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1403/2782] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:17:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:17:52.092544 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651315-61fc-4d9f-9ad4-ff83a21d46d8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:52.092833 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651315-61fc-4d9f-9ad4-ff83a21d46d8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:52.093098 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651315-61fc-4d9f-9ad4-ff83a21d46d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:52.113387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-6d651315-61fc-4d9f-9ad4-ff83a21d46d8 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:17:52.114581 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6d651315-61fc-4d9f-9ad4-ff83a21d46d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:52.172442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1380/2783] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:17:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:17:52.308558 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:17:52.310084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:17:52.392822 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d00ce-3d6e-48b8-8999-3dc34120b592 service glance] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:17:52.398073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:17:52.398690 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:17:52.398971 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:17:52.401939 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:17:52.408181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:17:52.409865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:17:52.418700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1404/2784] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:18:02.164868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=37 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:02.415695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=45 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:02.838996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:02.839332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:02.839614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:02.859989 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] MFA Rules not processed for user `9d20630980d642fa90636ff4c2d67075`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:18:02.861223 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:02.868017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1381/2785] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:18:03.126783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:03.128181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:03.184024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d6e74-63af-4be3-b44c-f550ed49de5e service swift] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:03.188630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:03.189248 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:03.189530 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:03.189806 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:03.191544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:03.201024 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:03.202620 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:03.210981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1405/2786] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:05.084580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:05.086122 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:05.154232 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df807c6b-5b30-4be6-93e4-c2a2bcdc1780 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:05.159748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:05.160373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:05.160655 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:05.160928 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:05.162652 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:05.174715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:05.176270 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ee612-9515-4e45-862e-dd736a707e94 service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:05.186033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1382/2787] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:06.033012 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:06.034443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:06.093409 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4139-ec27-4bfa-9728-367461b6b13d service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:06.096970 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:06.097583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:06.097860 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:06.098128 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:06.099851 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:06.168169 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:06.169625 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:06.179588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1406/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:06.529126 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:06.529624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:06.529741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:06.550516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:18:06.551720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:06.558988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1383/2789] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:18:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:18:06.571641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:06.583229 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf3f4-065e-40f2-aa01-c3c32b0a2b65 service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:06.588302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29958fd-5762-4d45-8f35-9561817dda28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:06.588941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29958fd-5762-4d45-8f35-9561817dda28 service swift] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:06.589223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29958fd-5762-4d45-8f35-9561817dda28 service swift] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:06.589498 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29958fd-5762-4d45-8f35-9561817dda28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:06.598487 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29958fd-5762-4d45-8f35-9561817dda28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:06.599981 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29958fd-5762-4d45-8f35-9561817dda28 service swift] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:06.608945 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1407/2790] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:18:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 18:18:16.544149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=38 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:16.605484 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=80 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:27.190530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:27.192191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:27.311518 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2291ed-1242-414e-bc27-64765c335437 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:27.322031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:27.322031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:27.322031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:27.322031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:27.322394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:27.337869 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:27.339988 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:27.357632 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1384/2791] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:27.974015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:27.974604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:27.974604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:27.996030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] MFA Rules not processed for user `f747a57b1a604347b9d2828bdddfa0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:18:27.997423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:28.005187 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1408/2792] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:18:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:18:28.315906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:28.325492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f48937-66bb-495e-afbc-f64cc8fdfa36 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:28.329963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:28.331350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:28.331350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:28.331350 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:28.332976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:28.342349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:28.343997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:28.352023 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1385/2793] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:37.989454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:38.348288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=40 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:42.891850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:42.901857 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9c49-d0bd-406d-b882-69b49c63558e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:42.906378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:42.907037 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:42.907317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:42.907586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:42.917452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:42.918879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:42.929201 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1409/2794] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:45.123820 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:45.126352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:45.142508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc453b9b-3ea0-491e-b21d-c58c708f165e service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:45.147983 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:45.148716 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:45.149046 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:45.149386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:45.151375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:45.164357 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:45.165936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:45.177443 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1386/2795] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:48.247454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:18:48.262979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd4e8f-0e62-4fee-89a1-c6617c37441a service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:18:48.266881 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:18:48.267700 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:18:48.268070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:18:48.268494 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:18:48.270391 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:18:48.285594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:18:48.287441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:18:48.302795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1410/2796] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:18:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:18:55.172579 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:18:58.294261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=12 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:19:49.658089 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:19:49.669115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eeb5b-37a4-472b-8992-2be61881055c service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:19:49.672897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:19:49.673516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:19:49.673792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:19:49.674103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:19:49.675953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:19:49.685738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:19:49.687152 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:19:49.695414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1387/2797] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:19:50.743345 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:19:50.751858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae54bb-3515-4e52-bfa5-961022c7a0f5 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:19:50.755439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:19:50.758311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:19:50.758311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:19:50.758311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:19:50.758311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:19:50.771258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:19:50.772683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:19:50.783305 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1411/2798] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:19:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:19:59.691690 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:20:00.778633 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:20:27.526394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:27.541802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f1402-4ed5-4995-9293-f7fd50a65523 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:27.547264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:27.547870 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:27.548149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:27.548459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:27.550186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:27.564145 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:27.565758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:27.577924 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1388/2799] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:20:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:20:34.007301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:34.016025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e02114-fbc5-4d52-bded-edc73bf8233c service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:34.019664 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:34.020259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:34.020572 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:34.020868 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:34.022595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:34.085361 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:34.086756 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:34.096108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1412/2800] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:20:37.464250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:37.466710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:37.484923 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a4d57-7341-464d-a78b-5b9cf1bd6eb8 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:37.490972 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:37.491747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:37.492028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:37.492298 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:37.494221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:37.508744 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:37.514012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:37.522855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1389/2801] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:20:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:20:37.663363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:37.664403 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:37.677735 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b550d4-bbb4-4bb9-acbf-4def130dcd9a service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:37.682748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:37.683382 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:37.683786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:37.684058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] PATH_INFO: `/v3/users/f747a57b1a604347b9d2828bdddfa0e9` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:37.685203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f747a57b1a604347b9d2828bdddfa0e9)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:37.687076 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:38.007872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f747a57b1a604347b9d2828bdddfa0e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.014744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f747a57b1a604347b9d2828bdddfa0e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.015415 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f747a57b1a604347b9d2828bdddfa0e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.016047 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f747a57b1a604347b9d2828bdddfa0e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.124496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-6e4f5b52-bd8f-4417-a7fc-4e6c83ea9d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f747a57b1a604347b9d2828bdddfa0e9'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.142927 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1413/2802] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:20:37 2022] DELETE /identity/v3/users/f747a57b1a604347b9d2828bdddfa0e9 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:20:38.377356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:38.388936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28fa02-ab4e-460e-91b9-6162414461cc service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:38.448799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:38.449432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:38.449806 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:38.450103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] PATH_INFO: `/v3/projects/1c789d40f8a84058a49533d382fe5303` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:38.451221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c789d40f8a84058a49533d382fe5303)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:38.453148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:38.566074 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c789d40f8a84058a49533d382fe5303', 'request_id': 'req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.633000 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c789d40f8a84058a49533d382fe5303', 'request_id': 'req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.640386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c789d40f8a84058a49533d382fe5303', 'request_id': 'req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.658947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] The token cache is being invalidate because project 1c789d40f8a84058a49533d382fe5303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:20:38.659239 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.659925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-16bbccce-7479-4cf0-bf3b-dfad2d8fc1b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:20:38.661519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1390/2803] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:20:38 2022] DELETE /identity/v3/projects/1c789d40f8a84058a49533d382fe5303 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:20:38.667316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:38.667812 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:38.667919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:38.687890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:20:38.689123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:38.747040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1414/2804] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:20:38.754940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:38.755274 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:38.755577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:38.775642 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:20:38.776875 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:38.784712 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1391/2805] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:20:38.789590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:38.799441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4457b-de40-4e64-93da-c67e97d902c9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:38.804599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:38.805342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:38.805626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:38.805901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:38.806550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:20:38.806853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:38.808449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:38.809879 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:20:38.823338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1415/2806] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:20:38.826293 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:38.837829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b4b63-595a-44a6-b83d-e19d31d0139b None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:38.843420 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:38.844027 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:38.844330 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:38.844614 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:38.845247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:38.847149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:38.925756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1392/2807] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:20:38 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:20:38.930562 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:38.940477 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa115d6-9a1c-446f-9493-ec6692cd6dc6 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:38.995460 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:38.996057 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:38.996356 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:38.996647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:38.997259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:38.999150 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:39.073655 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1416/2808] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:20:38 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:20:39.077730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:39.089073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee66cc1-be78-4acb-b03d-6f1c41d33eac admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:39.093982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:39.094612 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:39.094895 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:39.095169 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:39.095822 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:39.097444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:39.108154 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1393/2809] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:20:39.112115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:39.122134 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef21aa-5446-4707-8989-22c3b39ade95 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:39.126734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae51ecc-5adf-4778-9d47-84398a3cc450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:39.127320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae51ecc-5adf-4778-9d47-84398a3cc450 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:39.127600 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae51ecc-5adf-4778-9d47-84398a3cc450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:39.127872 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae51ecc-5adf-4778-9d47-84398a3cc450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:39.128532 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae51ecc-5adf-4778-9d47-84398a3cc450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:39.130092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae51ecc-5adf-4778-9d47-84398a3cc450 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:39.140513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1417/2810] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:20:39.144410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:39.160362 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c926fa-c140-42d7-a84b-38f67cd436e2 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:39.174505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:39.175173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:39.175455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:39.175729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] PATH_INFO: `/v3/projects/f516c7aef4f244e3a388146bc2a522f4/users/35b9e04e63bb43f9871bb8a3197adb5b/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:39.191456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f516c7aef4f244e3a388146bc2a522f4, user_id=35b9e04e63bb43f9871bb8a3197adb5b, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:39.194110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:39.297747 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:20:39.302399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1394/2811] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:20:39 2022] PUT /identity/v3/projects/f516c7aef4f244e3a388146bc2a522f4/users/35b9e04e63bb43f9871bb8a3197adb5b/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:20:39.310958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2273d18-42ce-4d70-92f1-e51781397fff None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:39.310958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2273d18-42ce-4d70-92f1-e51781397fff None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:39.310958 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2273d18-42ce-4d70-92f1-e51781397fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:39.333916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-f2273d18-42ce-4d70-92f1-e51781397fff None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:20:39.333916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-f2273d18-42ce-4d70-92f1-e51781397fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:39.389626 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1418/2812] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:20:39.399576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:39.401343 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:39.457625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cbd65-b1fd-4bcd-94cc-069812ec8103 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:39.461452 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:39.462055 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:39.462331 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:39.462658 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:39.471701 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:39.473142 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:39.482979 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1395/2813] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:20:44.970439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ce306-dff1-4e2a-9c7e-def1ff5dece0 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:44.970992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ce306-dff1-4e2a-9c7e-def1ff5dece0 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:44.970992 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ce306-dff1-4e2a-9c7e-def1ff5dece0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:45.004557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1f8ce306-dff1-4e2a-9c7e-def1ff5dece0 None None] MFA Rules not processed for user `35b9e04e63bb43f9871bb8a3197adb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:20:45.005794 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1f8ce306-dff1-4e2a-9c7e-def1ff5dece0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:45.075955 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1419/2814] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:20:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:20:45.092647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:45.094641 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:45.188990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625620a7-fddc-4d0d-a9f0-a6ba26c35d2e service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:45.195441 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:45.196382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:45.196664 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:45.197106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:45.205982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:45.207576 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:45.218318 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1396/2815] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:20:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:20:48.180331 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf396f-e3cb-4cfb-a993-c3ce1f5d69c1 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:48.180901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf396f-e3cb-4cfb-a993-c3ce1f5d69c1 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:48.180901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf396f-e3cb-4cfb-a993-c3ce1f5d69c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:48.188048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:20:48.202874 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-26bf396f-e3cb-4cfb-a993-c3ce1f5d69c1 None None] MFA Rules not processed for user `35b9e04e63bb43f9871bb8a3197adb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:20:48.204135 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-26bf396f-e3cb-4cfb-a993-c3ce1f5d69c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:48.211722 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1420/2816] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:20:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:20:48.954356 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:20:48.956530 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:49.028445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e412908-fa70-4fc2-93a6-81016ca2826a service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:20:49.033431 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:20:49.034030 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:20:49.034308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:20:49.034577 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:20:49.036297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:20:49.047964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:20:49.049555 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:20:49.059756 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1397/2817] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:20:58.587853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:20:59.055626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:21:16.193196 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:16.193676 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:16.193795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:16.213650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:21:16.214897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:16.270425 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1421/2818] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 1 18:21:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:21:16.311068 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:21:16.312536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:16.380297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf604a5-8d2d-4888-8685-d73c6caf2001 service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:21:16.386039 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:21:16.386635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:16.386911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:16.387179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:16.395084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:21:16.396671 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:21:16.405021 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1398/2819] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:21:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:21:20.588736 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:21:20.590376 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:20.605453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4938c-133d-41db-97f4-1d865fda8b87 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:21:20.610518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:21:20.611124 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:20.611398 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:20.611665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:20.613466 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:20.626018 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:21:20.627595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:21:20.638719 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1422/2820] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:21:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:21:24.304831 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:21:24.306536 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:24.321783 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f400ee7-1e3b-411c-8b12-80d3906278d7 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:21:24.326410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:21:24.327026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:24.327299 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:24.327564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] PATH_INFO: `/v3/users/9d20630980d642fa90636ff4c2d67075` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:24.328799 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d20630980d642fa90636ff4c2d67075)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:21:24.330682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:21:24.531753 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d20630980d642fa90636ff4c2d67075'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:24.539242 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d20630980d642fa90636ff4c2d67075'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:24.616436 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d20630980d642fa90636ff4c2d67075'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:24.617051 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d20630980d642fa90636ff4c2d67075'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:24.629843 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d20630980d642fa90636ff4c2d67075'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:24.635791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1399/2821] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:21:24 2022] DELETE /identity/v3/users/9d20630980d642fa90636ff4c2d67075 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:21:24.941699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:21:24.954404 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e9af7-2763-465a-b88e-194abfff91f4 service placement] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:21:25.011580 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:21:25.012202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:25.012510 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:25.012783 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] PATH_INFO: `/v3/projects/a2d986195f844099b8616c92bc2162b4` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:25.013878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d986195f844099b8616c92bc2162b4)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:21:25.015747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:21:25.119750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d986195f844099b8616c92bc2162b4', 'request_id': 'req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.188667 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d986195f844099b8616c92bc2162b4', 'request_id': 'req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.196278 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d986195f844099b8616c92bc2162b4', 'request_id': 'req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.208268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] The token cache is being invalidate because project a2d986195f844099b8616c92bc2162b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:21:25.211671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.211671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.211671 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1423/2822] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:21:24 2022] DELETE /identity/v3/projects/a2d986195f844099b8616c92bc2162b4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:21:25.215451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:21:25.216879 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:25.229966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc042b95-9f9a-4bd8-a464-384c4bb2f7a9 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:21:25.290491 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:21:25.291373 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:25.291647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:25.291916 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] PATH_INFO: `/v3/users/796f0da26d074d3f98e90da25c06f50d` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:25.293065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796f0da26d074d3f98e90da25c06f50d)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:21:25.294978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:21:25.297253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:21:25.474775 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796f0da26d074d3f98e90da25c06f50d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.481470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796f0da26d074d3f98e90da25c06f50d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.580964 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796f0da26d074d3f98e90da25c06f50d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.581593 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796f0da26d074d3f98e90da25c06f50d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.594158 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-12795e3f-ec56-4c3e-b853-cd481067edc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796f0da26d074d3f98e90da25c06f50d'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:25.599974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1400/2823] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:21:25 2022] DELETE /identity/v3/users/796f0da26d074d3f98e90da25c06f50d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:21:25.942542 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:21:25.944131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:21:25.958294 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201ecbc-4287-4877-b4f8-8a3032a5b7aa admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:21:25.977160 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:21:26.015181 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:21:26.015748 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:21:26.016266 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:21:26.016649 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] PATH_INFO: `/v3/projects/3bbf1e4db6d445519c083ac433d6bd1d` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:21:26.018074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bbf1e4db6d445519c083ac433d6bd1d)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:21:26.020074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:21:26.156744 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bbf1e4db6d445519c083ac433d6bd1d', 'request_id': 'req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:26.232782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bbf1e4db6d445519c083ac433d6bd1d', 'request_id': 'req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:26.238889 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bbf1e4db6d445519c083ac433d6bd1d', 'request_id': 'req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:26.250360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] The token cache is being invalidate because project 3bbf1e4db6d445519c083ac433d6bd1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:21:26.250639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:26.251288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:21:26.252885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1424/2824] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:21:25 2022] DELETE /identity/v3/projects/3bbf1e4db6d445519c083ac433d6bd1d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:21:35.596522 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:21:36.247307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:22:32.456438 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:22:32.457026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:22:32.457191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:22:32.486288 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] MFA Rules not processed for user `35b9e04e63bb43f9871bb8a3197adb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:22:32.487486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:22:32.557445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1401/2825] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:22:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:22:42.554860 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=37 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:23:14.397183 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:23:14.402221 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:23:14.530408 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ab696-14aa-4c1f-a7e8-7e9050673eb0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:23:14.535601 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:23:14.536259 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:23:14.536566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:23:14.536839 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:23:14.538586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:23:14.551175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:23:14.553108 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473527bc-2eda-4f72-87cf-fe369fa85328 service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:23:14.562840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1425/2826] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:23:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:23:24.558589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=59 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:24:03.732915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:24:03.734303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:03.790741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079c3c-d11e-421b-aecc-55987c0fbac2 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:24:03.794595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:24:03.795198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:24:03.795479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:24:03.795752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:24:03.797564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:03.887796 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:24:03.889173 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:24:03.914885 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1402/2827] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:24:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:24:13.912186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=66 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:24:20.431118 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:24:20.431668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:24:20.431668 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:24:20.450592 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] MFA Rules not processed for user `35b9e04e63bb43f9871bb8a3197adb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:24:20.451789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:20.458878 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1426/2828] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:24:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:24:20.766802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:24:20.768386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:20.846527 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3d2f-44d3-469e-84f9-93b4495b6cb8 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:24:20.850759 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:24:20.851354 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:24:20.851630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:24:20.851899 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:24:20.853644 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:20.862956 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:24:20.864502 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:24:20.872340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1403/2829] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:24:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:24:30.445194 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:24:30.868805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=35 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:24:30.909360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:24:30.910990 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:30.926797 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed62ce-244d-40ea-89a2-561502f64827 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:24:30.932519 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:24:30.933156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:24:30.933437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:24:30.933730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:24:30.935452 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:30.948156 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:24:30.949774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:24:30.960524 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1427/2830] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:24:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:24:35.469367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:24:35.470815 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:35.486963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c85f7-5d73-4aea-930e-32b9dd8075f5 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:24:35.493505 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:24:35.494106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:24:35.494389 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:24:35.494657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:24:35.496387 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:24:35.509217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:24:35.510786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:24:35.521968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1404/2831] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:24:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:24:40.956483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:24:45.517123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:25:19.109711 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:25:19.119240 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8573-2c37-415c-9ef0-f88d1f510b7a service nova] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:25:19.123162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:25:19.123795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:25:19.124073 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:25:19.124381 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:25:19.126151 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:25:19.136163 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:25:19.137604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:25:19.145595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1428/2832] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:25:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:25:29.141856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=6 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:25:50.473622 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:25:50.484946 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f6114-736d-4a46-a80a-2e8a021bd474 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:25:50.490591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:25:50.491432 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:25:50.491619 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:25:50.491748 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:25:50.493497 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:25:50.504264 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:25:50.505884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:25:50.514832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1405/2833] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:25:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:26:00.510681 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=6 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:26:01.923796 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:26:01.933234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a66b0a-07c6-440f-984e-02b40d6751dd service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:26:01.937121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:26:01.937785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:26:01.938092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:26:01.938426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:26:01.940172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:26:01.954179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:26:01.958063 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37c1a2-fa5b-490e-81a3-274690883651 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:26:01.967120 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1429/2834] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:26:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:26:07.217468 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:26:07.230269 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbfdcf-c5f8-4c58-bef8-e5375f9b21f7 service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:26:07.236095 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dec0395ef74fd5a6023ae06a1b9538', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:26:07.236661 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:26:07.236954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:26:07.237229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:26:07.239043 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:26:07.254026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:26:07.255585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:26:07.267691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1406/2835] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:26:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:26:11.913482 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:26:11.914147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:26:11.914147 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:26:11.934074 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] MFA Rules not processed for user `35b9e04e63bb43f9871bb8a3197adb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:26:11.935277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:26:11.942329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1430/2836] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:26:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:26:12.631837 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:26:12.644012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073720-1407-41a4-ba91-91bc4af731ae service nova] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:26:12.649361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:26:12.649969 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:26:12.650250 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:26:12.650523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:26:12.652352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:26:12.664294 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:26:12.666244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:26:12.676940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1407/2837] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:26:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:26:21.928427 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=9 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:26:22.672531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=12 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:02.550256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:02.559847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db347fa-8f2f-4b52-a5ab-d4e72f699b41 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:02.563890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:02.564523 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:02.564800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:02.565070 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:02.566757 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:02.650908 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:02.652366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:02.671114 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1431/2838] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:06.040103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:06.040103 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:06.064191 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336f196-8254-4e5e-bd00-1a5600371bae service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:06.070017 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:06.070741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:06.071257 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:06.071599 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:06.073680 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:06.086084 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:06.087682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:06.099340 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1408/2839] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:06.158311 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:06.159733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:06.172164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfce1a-f47d-42da-8df3-d579127c111d service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:06.176924 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:06.177606 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:06.177886 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:06.178159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] PATH_INFO: `/v3/users/35b9e04e63bb43f9871bb8a3197adb5b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:06.179277 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b9e04e63bb43f9871bb8a3197adb5b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:06.181256 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:06.382777 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b9e04e63bb43f9871bb8a3197adb5b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.389539 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b9e04e63bb43f9871bb8a3197adb5b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.390192 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b9e04e63bb43f9871bb8a3197adb5b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.390795 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b9e04e63bb43f9871bb8a3197adb5b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.473559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-274e3c0b-089b-455f-bac2-296c99ec6aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b9e04e63bb43f9871bb8a3197adb5b'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.491787 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1432/2840] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:06 2022] DELETE /identity/v3/users/35b9e04e63bb43f9871bb8a3197adb5b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:06.707549 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:06.719359 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786437e7-41f2-492f-89d7-871b9a33df1b service placement] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:06.780380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:06.781011 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:06.781547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:06.781825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] PATH_INFO: `/v3/projects/f516c7aef4f244e3a388146bc2a522f4` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:06.782934 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f516c7aef4f244e3a388146bc2a522f4)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:06.784902 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:06.894131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f516c7aef4f244e3a388146bc2a522f4', 'request_id': 'req-8689e6b2-07a2-4dc4-a188-0191e700b1f8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.968859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f516c7aef4f244e3a388146bc2a522f4', 'request_id': 'req-8689e6b2-07a2-4dc4-a188-0191e700b1f8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.974564 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f516c7aef4f244e3a388146bc2a522f4', 'request_id': 'req-8689e6b2-07a2-4dc4-a188-0191e700b1f8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.986347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] The token cache is being invalidate because project f516c7aef4f244e3a388146bc2a522f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:06.986660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.987277 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-8689e6b2-07a2-4dc4-a188-0191e700b1f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:06.988782 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1409/2841] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:06 2022] DELETE /identity/v3/projects/f516c7aef4f244e3a388146bc2a522f4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:06.993578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:06.993868 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:06.994172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.048513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:07.051309 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:07.052610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:07.109231 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1433/2842] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:07.117144 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.117638 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.117741 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.136811 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:07.138504 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:07.146295 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1410/2843] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:07.150638 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.160802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add50730-eb5f-40e0-9960-73ae311dac58 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.165359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.165953 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.166234 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.166506 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.167152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:28:07.167453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.169426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.181432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1434/2844] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:28:07.185710 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.194948 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc340c02-efa0-4642-9f89-63035a2075e6 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.199616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.200210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.201063 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.201338 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.201965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.203842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.296595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1411/2845] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:07 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:07.301059 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.311132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e14-6d3c-4825-9f03-22f015a8aa13 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.369213 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.369829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.370107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.370378 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.371015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.372976 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.464576 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1435/2846] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:07 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:28:07.469186 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.479859 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e0bf-0998-499d-a642-606301b57aa6 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.484898 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.485616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.489765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.489765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.489765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.489765 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.498914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1412/2847] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:07.503009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.513677 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2b77-bfd9-41ab-b92e-c99f79225eca admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.518699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e380c-4310-4022-88dc-6b7fb8e2b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.519288 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e380c-4310-4022-88dc-6b7fb8e2b9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.519564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e380c-4310-4022-88dc-6b7fb8e2b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.519833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e380c-4310-4022-88dc-6b7fb8e2b9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.520595 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e380c-4310-4022-88dc-6b7fb8e2b9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.522164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e380c-4310-4022-88dc-6b7fb8e2b9b1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.532982 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1436/2848] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:07.538304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.548674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ec1b3-d1b7-4da4-ba79-0ceac5ba3a47 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.553234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.553823 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.554099 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.555442 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] PATH_INFO: `/v3/projects/ad61357739444221aeff65f318a66c86/users/cf5e944a7d9e442db575a25045f9b725/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.570514 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad61357739444221aeff65f318a66c86, user_id=cf5e944a7d9e442db575a25045f9b725, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.574044 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.675649 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1413/2849] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:07 2022] PUT /identity/v3/projects/ad61357739444221aeff65f318a66c86/users/cf5e944a7d9e442db575a25045f9b725/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:28:07.687186 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.687469 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.687737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.715340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] MFA Rules not processed for user `cf5e944a7d9e442db575a25045f9b725`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:07.716730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:07.779129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1437/2850] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:07.783570 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.789177 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:07.846207 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d91c0e-00f7-40e8-b8b7-2ea89728be85 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:07.850800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:07.851390 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:07.854346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:07.854346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:07.854346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:07.854654 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:07.962276 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1414/2851] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:07 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:07.966394 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:07.971916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:07.977583 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ff70-756e-42bd-abfd-7e0169cf626e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:08.033769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:08.034360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.034637 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.034907 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.035533 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:08.037439 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:08.115480 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1438/2852] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:07 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:28:08.119307 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:08.130496 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1921a-017b-4469-b314-b8d88c10970d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:08.135112 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:08.135703 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.135978 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.136243 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.136912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:08.138864 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:08.149247 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1415/2853] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:08.155332 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:08.165344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee3c6-2f2a-47ba-81fd-375f67c80f4c admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:08.170570 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:08.171264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.171547 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.172107 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] PATH_INFO: `/v3/projects/a6fede798a544919b0ca2914ad01a7dc/users/a1d65697eb5d4ceba571a0207b3ada79/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.187009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fede798a544919b0ca2914ad01a7dc, user_id=a1d65697eb5d4ceba571a0207b3ada79, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:08.189659 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:08.249865 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1439/2854] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:08 2022] PUT /identity/v3/projects/a6fede798a544919b0ca2914ad01a7dc/users/a1d65697eb5d4ceba571a0207b3ada79/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:08.253584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:08.263738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d1228-f07a-4e9b-babe-2cfe15f0d655 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:08.329303 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689473f-726d-4b9a-9548-1feed9bd1fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:08.329907 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689473f-726d-4b9a-9548-1feed9bd1fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.330187 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689473f-726d-4b9a-9548-1feed9bd1fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.330461 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689473f-726d-4b9a-9548-1feed9bd1fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.331105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689473f-726d-4b9a-9548-1feed9bd1fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:08.332754 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689473f-726d-4b9a-9548-1feed9bd1fa1 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:08.342936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1416/2855] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:08.347152 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:08.358286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985ef0-1144-4379-9c24-e51ce04c58dd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:08.363255 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:08.363912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.364189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.364509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] PATH_INFO: `/v3/projects/a6fede798a544919b0ca2914ad01a7dc/users/a1d65697eb5d4ceba571a0207b3ada79/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.366550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fede798a544919b0ca2914ad01a7dc, user_id=a1d65697eb5d4ceba571a0207b3ada79, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:08.369205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:08.379335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1440/2856] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:08 2022] PUT /identity/v3/projects/a6fede798a544919b0ca2914ad01a7dc/users/a1d65697eb5d4ceba571a0207b3ada79/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:08.389131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.389421 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.389745 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.417793 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] MFA Rules not processed for user `a1d65697eb5d4ceba571a0207b3ada79`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:08.418992 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:08.483093 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:08.486584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1417/2857] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 18:28:08.500730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:08.500730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:08.579122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b295eaf-a9b9-4639-8394-acdcfa31ce4e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:08.582961 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce3e57-1a78-40f1-ae3e-7c52f9465eed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:08.583564 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce3e57-1a78-40f1-ae3e-7c52f9465eed service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:08.583845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce3e57-1a78-40f1-ae3e-7c52f9465eed service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:08.584121 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce3e57-1a78-40f1-ae3e-7c52f9465eed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:08.593830 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce3e57-1a78-40f1-ae3e-7c52f9465eed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:08.595275 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce3e57-1a78-40f1-ae3e-7c52f9465eed service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:08.604639 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1441/2858] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:09.544660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:09.607394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08e01-d160-49d4-bfe7-5c8a2561d232 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:09.611790 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:09.612444 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:09.612719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:09.612987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] PATH_INFO: `/v3/users/cf5e944a7d9e442db575a25045f9b725` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:09.614091 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5e944a7d9e442db575a25045f9b725)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:09.616033 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:09.805906 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5e944a7d9e442db575a25045f9b725'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:09.812729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5e944a7d9e442db575a25045f9b725'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:09.873306 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5e944a7d9e442db575a25045f9b725'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:09.873945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5e944a7d9e442db575a25045f9b725'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:09.886917 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5e944a7d9e442db575a25045f9b725'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:09.893348 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1418/2859] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:09 2022] DELETE /identity/v3/users/cf5e944a7d9e442db575a25045f9b725 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:09.897856 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:09.898352 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:09.898461 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:09.919083 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:09.921368 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:09.953384 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:09.979342 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1442/2860] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:09.989720 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:09.991123 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:10.048968 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519dc3-4364-4244-b936-4d94c718d52e admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:10.052800 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:10.053410 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:10.053687 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:10.053954 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:10.063171 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:10.064560 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:10.073726 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1419/2861] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:10.502932 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:10.504428 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:10.518731 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c9f71-1a72-46cd-b93b-19c712b0617f None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:10.528349 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:10.529454 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:10.529938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:10.530495 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] PATH_INFO: `/v3/projects/ad61357739444221aeff65f318a66c86` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:10.532228 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad61357739444221aeff65f318a66c86)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:10.534081 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:10.665994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad61357739444221aeff65f318a66c86', 'request_id': 'req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:10.746172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad61357739444221aeff65f318a66c86', 'request_id': 'req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:10.752212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad61357739444221aeff65f318a66c86', 'request_id': 'req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:10.763552 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] The token cache is being invalidate because project ad61357739444221aeff65f318a66c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:10.763840 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:10.764502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:10.766058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1443/2862] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:10 2022] DELETE /identity/v3/projects/ad61357739444221aeff65f318a66c86 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:10.770097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:10.780711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e1caf-a445-4dd0-9ca8-bc317e5fab9f service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:10.799586 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:10.837210 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:10.837832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:10.838113 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:10.838402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] PATH_INFO: `/v3/users/a1d65697eb5d4ceba571a0207b3ada79` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:10.839492 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d65697eb5d4ceba571a0207b3ada79)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:10.842077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:11.054413 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d65697eb5d4ceba571a0207b3ada79'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.061253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d65697eb5d4ceba571a0207b3ada79'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.120742 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d65697eb5d4ceba571a0207b3ada79'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.121372 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d65697eb5d4ceba571a0207b3ada79'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.133434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-610051ba-0e95-4bca-a38e-a0c1d135dc3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d65697eb5d4ceba571a0207b3ada79'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.139760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1420/2863] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:10 2022] DELETE /identity/v3/users/a1d65697eb5d4ceba571a0207b3ada79 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:11.517714 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:11.519189 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:11.533898 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0eb2-24a4-4523-950c-1d2b1a5ddce0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:11.594126 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:11.594686 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:11.594979 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:11.595292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] PATH_INFO: `/v3/projects/a6fede798a544919b0ca2914ad01a7dc` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:11.596441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6fede798a544919b0ca2914ad01a7dc)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:11.599994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:11.692935 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fede798a544919b0ca2914ad01a7dc', 'request_id': 'req-7a839f7a-536c-4fcb-a32e-378ad410df6f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.783559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fede798a544919b0ca2914ad01a7dc', 'request_id': 'req-7a839f7a-536c-4fcb-a32e-378ad410df6f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.788282 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:11.789613 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fede798a544919b0ca2914ad01a7dc', 'request_id': 'req-7a839f7a-536c-4fcb-a32e-378ad410df6f'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.801833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] The token cache is being invalidate because project a6fede798a544919b0ca2914ad01a7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:11.802146 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.802774 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-7a839f7a-536c-4fcb-a32e-378ad410df6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:11.804251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1444/2864] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:11 2022] DELETE /identity/v3/projects/a6fede798a544919b0ca2914ad01a7dc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:11.809369 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:11.810543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:11.810543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:11.829629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:11.830813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:11.890231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1421/2865] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:11.897834 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:11.898116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:11.898396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:11.918317 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:11.920036 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:11.928265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1445/2866] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:11.932553 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:11.942930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139460ab-2a58-4d36-9f54-11f18fa92558 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:11.947563 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:11.948157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:11.948463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:11.948738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] PATH_INFO: `/v3/domains` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:11.949383 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:28:11.949679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:11.951301 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:11.962604 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1422/2867] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:28:11.967438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:11.976871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50ea32-959e-48b5-81a4-89d53cb63fec None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:11.981538 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:11.982750 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:11.983030 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:11.983303 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:11.983943 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:11.985862 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.070651 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1446/2868] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:11 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:28:12.076591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.086567 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9ce4-00fd-4a7a-806b-22c532ef1385 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.113370 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:12.143253 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.143850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.144134 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.144447 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.145081 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.146976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.253190 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1423/2869] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:12 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:12.257458 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.267640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d2ffd-4c93-4ccf-ad5f-ebc7e5c95b92 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.272437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.273038 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.273316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.273648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.274293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.276106 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.286850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1447/2870] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 18:28:12.290739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.301582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cec2-7a6a-461e-872c-4c46c863f7ac admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.306592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4bab0-3419-4bc8-94ca-526954e57ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.307179 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bab0-3419-4bc8-94ca-526954e57ada admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.307455 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bab0-3419-4bc8-94ca-526954e57ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.307723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bab0-3419-4bc8-94ca-526954e57ada admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.308414 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4bab0-3419-4bc8-94ca-526954e57ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.310525 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4bab0-3419-4bc8-94ca-526954e57ada admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.320221 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1424/2871] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:12.324159 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.334609 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac767c-fcc3-4893-aa29-bd1ea4fe13f2 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.339173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.339810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.340088 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.340385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] PATH_INFO: `/v3/projects/670f02b3a076479d8bb9a9f8185470e2/users/9cdc080c2e3240348f13952f4d5ad7d8/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.358389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] RBAC: Authorizing `identity:create_grant(project_id=670f02b3a076479d8bb9a9f8185470e2, user_id=9cdc080c2e3240348f13952f4d5ad7d8, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.361082 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.429172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1448/2872] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:12 2022] PUT /identity/v3/projects/670f02b3a076479d8bb9a9f8185470e2/users/9cdc080c2e3240348f13952f4d5ad7d8/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:12.440679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.440965 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.441236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.470237 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] MFA Rules not processed for user `9cdc080c2e3240348f13952f4d5ad7d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:12.471584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:12.533850 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1425/2873] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:12.538443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.599330 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117cecf-7d19-4337-a995-905b0e66795b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.603902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.604529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.604808 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.605078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.605709 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.607763 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.693742 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1449/2874] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:12 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:12.697855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.709189 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b5c7-580c-46fd-aeed-9c7ebdec6f48 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.766465 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.767067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.767347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.767621 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.768258 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.770214 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.850268 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1426/2875] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:12 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:12.852849 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.863669 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a363d10-32e2-412b-a3c3-6a36a42efcbf admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.868437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.869039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.869315 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.869589 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.870227 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.871825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.883155 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1450/2876] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:12.887304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:12.898062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776b88f-31a3-41b9-b61f-c9cf7d755f2f admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:12.902730 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:12.903912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:12.904199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:12.904489 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] PATH_INFO: `/v3/projects/554b167c6cfd40beb260b1951d2dbcb3/users/487fc1276cb545c9ba1e8a3d8ef9bb90/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:12.919944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=554b167c6cfd40beb260b1951d2dbcb3, user_id=487fc1276cb545c9ba1e8a3d8ef9bb90, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:12.922980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:12.997936 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1427/2877] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:12 2022] PUT /identity/v3/projects/554b167c6cfd40beb260b1951d2dbcb3/users/487fc1276cb545c9ba1e8a3d8ef9bb90/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:28:13.001737 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:13.012017 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a3b9f-9e08-448a-9ff3-29114c6d4cfc admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:13.067873 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061755c-46cf-4fd9-8ff9-63594856fa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:13.068500 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061755c-46cf-4fd9-8ff9-63594856fa8a admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:13.068782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061755c-46cf-4fd9-8ff9-63594856fa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:13.069071 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061755c-46cf-4fd9-8ff9-63594856fa8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:13.069695 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061755c-46cf-4fd9-8ff9-63594856fa8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:13.071261 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061755c-46cf-4fd9-8ff9-63594856fa8a admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:13.073279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:13.081734 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1451/2878] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:13.085828 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:13.091866 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:13.097244 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b7b5-ba60-46ae-9abf-3683436f9e15 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:13.101913 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:13.102557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:13.102832 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:13.103102 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] PATH_INFO: `/v3/projects/554b167c6cfd40beb260b1951d2dbcb3/users/487fc1276cb545c9ba1e8a3d8ef9bb90/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:13.105078 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=554b167c6cfd40beb260b1951d2dbcb3, user_id=487fc1276cb545c9ba1e8a3d8ef9bb90, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:13.107852 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:13.117915 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1428/2879] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:13 2022] PUT /identity/v3/projects/554b167c6cfd40beb260b1951d2dbcb3/users/487fc1276cb545c9ba1e8a3d8ef9bb90/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:13.128132 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:13.128457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:13.128761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:13.156833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] MFA Rules not processed for user `487fc1276cb545c9ba1e8a3d8ef9bb90`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:13.158026 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:13.226586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1452/2880] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:13.237575 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:13.239631 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:13.299729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14125a03-de1c-48a0-9d0f-86fecd62e7e8 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:13.303498 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:13.304240 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:13.306511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:13.306511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:13.314287 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:13.315707 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:13.326217 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1429/2881] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:28:13.437363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:13.446902 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb751c-533b-4180-8cbc-eacaf8a8dec2 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:13.452351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4e73f-c210-473e-9feb-6a56ef0e5d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:13.452351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4e73f-c210-473e-9feb-6a56ef0e5d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:13.452351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4e73f-c210-473e-9feb-6a56ef0e5d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:13.452351 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4e73f-c210-473e-9feb-6a56ef0e5d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:13.508514 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4e73f-c210-473e-9feb-6a56ef0e5d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:13.509977 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4e73f-c210-473e-9feb-6a56ef0e5d16 service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:13.519168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1453/2882] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:16.747124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:16.808040 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0b4de-96a6-491e-a97f-fb528ba61eef service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:16.812685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:16.813304 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:16.813582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:16.813853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] PATH_INFO: `/v3/users/9cdc080c2e3240348f13952f4d5ad7d8` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:16.814974 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cdc080c2e3240348f13952f4d5ad7d8)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:16.816921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:17.033896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cdc080c2e3240348f13952f4d5ad7d8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.040625 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cdc080c2e3240348f13952f4d5ad7d8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.117678 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cdc080c2e3240348f13952f4d5ad7d8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.118297 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cdc080c2e3240348f13952f4d5ad7d8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.130724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cdc080c2e3240348f13952f4d5ad7d8'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.136801 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1430/2883] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:16 2022] DELETE /identity/v3/users/9cdc080c2e3240348f13952f4d5ad7d8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:17.141885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:17.142358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:17.142481 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:17.162232 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:17.163432 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:17.221102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1454/2884] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:17.230739 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:17.232143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:17.245786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:17.290819 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800375d9-edb2-449e-bf05-5ea2849a192d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:17.294426 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:17.295228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:17.295565 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:17.295830 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:17.307689 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:17.309165 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:17.320440 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1431/2885] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:17.329743 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:17.331286 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:17.357843 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=100 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:17.401707 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7371-58a8-4403-8db7-6d5574b03fd3 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:17.407405 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:17.408046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:17.408586 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:17.409042 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:17.411168 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:17.423610 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:17.425206 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:17.436417 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1455/2886] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:17.762459 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:17.764531 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:17.779961 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7a9b1-4164-4732-adf5-d0bcdba01119 service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:17.785256 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:17.786056 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:17.786332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:17.786588 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] PATH_INFO: `/v3/projects/670f02b3a076479d8bb9a9f8185470e2` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:17.787708 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] RBAC: Authorizing `identity:delete_project(project_id=670f02b3a076479d8bb9a9f8185470e2)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:17.789674 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:17.894862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670f02b3a076479d8bb9a9f8185470e2', 'request_id': 'req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.975069 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670f02b3a076479d8bb9a9f8185470e2', 'request_id': 'req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.981228 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670f02b3a076479d8bb9a9f8185470e2', 'request_id': 'req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.992987 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] The token cache is being invalidate because project 670f02b3a076479d8bb9a9f8185470e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:17.993275 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.993903 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:17.995380 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1432/2887] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:17 2022] DELETE /identity/v3/projects/670f02b3a076479d8bb9a9f8185470e2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:18.000116 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:18.010474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567730cb-3569-4a0e-87cd-62293bc1f524 service placement] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:18.066175 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:18.066799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:18.067075 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:18.070831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] PATH_INFO: `/v3/users/487fc1276cb545c9ba1e8a3d8ef9bb90` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:18.070831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=487fc1276cb545c9ba1e8a3d8ef9bb90)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:18.070831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:18.257077 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487fc1276cb545c9ba1e8a3d8ef9bb90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:18.264531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487fc1276cb545c9ba1e8a3d8ef9bb90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:18.265197 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487fc1276cb545c9ba1e8a3d8ef9bb90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:18.265803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487fc1276cb545c9ba1e8a3d8ef9bb90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:18.354782 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-3616b8fb-0bda-49dc-b0ac-e42419a91caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487fc1276cb545c9ba1e8a3d8ef9bb90'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:18.380789 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1456/2888] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:17 2022] DELETE /identity/v3/users/487fc1276cb545c9ba1e8a3d8ef9bb90 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:18.799971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:18.801472 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:18.814967 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac281d-f4d4-4d56-a4ed-e729b4f4704a admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:18.875346 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:18.875970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:18.876248 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:18.876547 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] PATH_INFO: `/v3/projects/554b167c6cfd40beb260b1951d2dbcb3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:18.877650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=554b167c6cfd40beb260b1951d2dbcb3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:18.879697 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.025529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:19.032241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '554b167c6cfd40beb260b1951d2dbcb3', 'request_id': 'req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:19.120620 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '554b167c6cfd40beb260b1951d2dbcb3', 'request_id': 'req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:19.126944 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '554b167c6cfd40beb260b1951d2dbcb3', 'request_id': 'req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:19.138183 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] The token cache is being invalidate because project 554b167c6cfd40beb260b1951d2dbcb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:19.138477 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:19.139376 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-b40f66ea-e0b8-406e-86f1-d4f50ad807c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:19.140889 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1433/2889] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:18 2022] DELETE /identity/v3/projects/554b167c6cfd40beb260b1951d2dbcb3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:19.145807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.146092 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.146363 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.166025 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:19.167225 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:19.193802 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:19.226850 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1457/2890] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:19.235166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.235446 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.235719 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.256231 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:19.257456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:19.265770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1434/2891] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:19.270085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.279529 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab487ae3-2fbe-4ccf-b550-7e5e4985d8d9 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.284173 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.284809 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.285090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.285359 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.286007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:28:19.286307 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.287876 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.300344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1458/2892] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:28:19.304534 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.314065 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbf7cb-1b56-4242-8b49-63507d2c068c None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.318596 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.319181 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.319458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.319723 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.320368 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.322241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.422398 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1435/2893] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:19 2022] POST /identity/v3/projects => generated 407 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:19.426578 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.437300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0cd27-632c-4e42-a4f0-ef29e8ebea3b admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.497591 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.498410 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.498754 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.499078 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.499721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.501831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.586612 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1459/2894] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:19 2022] POST /identity/v3/users => generated 304 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:19.590774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.601543 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675596-9d24-4637-a47a-1d50082cc416 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.606131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.606721 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.606996 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.607940 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.608634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.610203 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.620148 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1436/2895] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:19.623909 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.633832 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14ef65-2d15-4723-8392-d855ad9036dd admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.638423 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cca86-e716-4e8b-876b-10e853cc2be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.639015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cca86-e716-4e8b-876b-10e853cc2be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.639293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cca86-e716-4e8b-876b-10e853cc2be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.639566 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cca86-e716-4e8b-876b-10e853cc2be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.640217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cca86-e716-4e8b-876b-10e853cc2be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.641800 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cca86-e716-4e8b-876b-10e853cc2be2 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.652648 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1460/2896] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:19.657131 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.666914 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5da378-7754-46f5-a24b-5953f14092c4 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.672292 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.673010 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.673235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.673506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] PATH_INFO: `/v3/projects/0c061a803df6446a841147e2649ca62b/users/d82f94f8f81e4f2485f87b1850dd56aa/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.687542 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c061a803df6446a841147e2649ca62b, user_id=d82f94f8f81e4f2485f87b1850dd56aa, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.690174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:19.778151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1437/2897] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:19 2022] PUT /identity/v3/projects/0c061a803df6446a841147e2649ca62b/users/d82f94f8f81e4f2485f87b1850dd56aa/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:19.790202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.790486 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.790762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.824141 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] MFA Rules not processed for user `d82f94f8f81e4f2485f87b1850dd56aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:19.825389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:19.888448 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1461/2898] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:19.893557 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:19.955791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896758e-f58d-4a94-b8e2-654318f1c35d admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:19.960260 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:19.960880 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:19.961156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:19.961427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:19.962067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:19.963952 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.055509 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1438/2899] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:19 2022] POST /identity/v3/projects => generated 407 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:20.059594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:20.070209 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a285d-d532-4798-8607-e37dadc99134 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:20.094474 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:20.125814 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:20.126420 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.126699 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.126972 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.127641 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:20.129944 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.260833 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1462/2900] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:20 2022] POST /identity/v3/users => generated 310 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:28:20.265735 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:20.276066 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab7b7-af1b-4a96-8f5e-fe8857915c2c admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:20.280953 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:20.281573 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.281855 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.282129 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.282774 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:20.284394 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.294839 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1439/2901] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:20.299249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:20.309177 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46acbe-99e4-4992-8da5-328094eb7288 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:20.314340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:20.315046 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.315323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.315593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] PATH_INFO: `/v3/projects/c2e05239924444beb61a5462d2c25ee1/users/e94119d1611b45c09c4953fcc92b0af4/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.331167 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e05239924444beb61a5462d2c25ee1, user_id=e94119d1611b45c09c4953fcc92b0af4, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:20.333811 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.412443 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1463/2902] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:20 2022] PUT /identity/v3/projects/c2e05239924444beb61a5462d2c25ee1/users/e94119d1611b45c09c4953fcc92b0af4/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:20.416508 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:20.426980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9ed2f-e6b8-4d65-a7b8-bd6b641d7230 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:20.461266 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:20.483031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfa280-ae28-43e4-889b-af8a4fff35db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:20.483634 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfa280-ae28-43e4-889b-af8a4fff35db admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.483912 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfa280-ae28-43e4-889b-af8a4fff35db admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.484184 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfa280-ae28-43e4-889b-af8a4fff35db admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.484927 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfa280-ae28-43e4-889b-af8a4fff35db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:20.486836 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfa280-ae28-43e4-889b-af8a4fff35db admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.496743 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1440/2903] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:20.501823 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:20.512265 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99c85a-a272-41df-be9f-0942207dc126 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:20.517513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:20.520007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.520007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.520007 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] PATH_INFO: `/v3/projects/c2e05239924444beb61a5462d2c25ee1/users/e94119d1611b45c09c4953fcc92b0af4/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.520761 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e05239924444beb61a5462d2c25ee1, user_id=e94119d1611b45c09c4953fcc92b0af4, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:20.523338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.533575 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1464/2904] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:20 2022] PUT /identity/v3/projects/c2e05239924444beb61a5462d2c25ee1/users/e94119d1611b45c09c4953fcc92b0af4/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:20.544029 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.544333 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.545603 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.573235 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] MFA Rules not processed for user `e94119d1611b45c09c4953fcc92b0af4`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:20.574445 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:20.642528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1441/2905] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:20.653323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:20.655237 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:20.712004 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707f79-54cb-446d-be90-4af3513279ed admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:20.715674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:20.716272 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:20.716630 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:20.716963 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:20.725845 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:20.727280 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:20.737273 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1465/2906] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:20 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:21.132097 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:21.142473 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d421-9e43-4d42-9073-0964af54e1e7 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:21.146075 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525fd62-a13d-451b-8570-4fccdbe7fbe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:21.149115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525fd62-a13d-451b-8570-4fccdbe7fbe2 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:21.149115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525fd62-a13d-451b-8570-4fccdbe7fbe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:21.149115 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525fd62-a13d-451b-8570-4fccdbe7fbe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:21.203077 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525fd62-a13d-451b-8570-4fccdbe7fbe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:21.204685 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525fd62-a13d-451b-8570-4fccdbe7fbe2 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:21.213512 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1442/2907] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:30.732385 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=58,keystone:INSERT=5 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:31.209519 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=74 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:32.597804 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:32.659640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d87a0-4ac1-499d-a383-e4a4c7d0267e service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:32.664115 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:32.664785 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:32.665064 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:32.665333 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] PATH_INFO: `/v3/users/d82f94f8f81e4f2485f87b1850dd56aa` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:32.666437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82f94f8f81e4f2485f87b1850dd56aa)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:32.668320 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:32.871084 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82f94f8f81e4f2485f87b1850dd56aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:32.877994 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82f94f8f81e4f2485f87b1850dd56aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:32.878657 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82f94f8f81e4f2485f87b1850dd56aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:32.879268 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82f94f8f81e4f2485f87b1850dd56aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:32.943316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82f94f8f81e4f2485f87b1850dd56aa'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:32.961016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1466/2908] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:32 2022] DELETE /identity/v3/users/d82f94f8f81e4f2485f87b1850dd56aa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:32.965679 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:32.966245 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:32.966347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:32.986479 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:32.987684 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:33.045052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1443/2909] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:33.055803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:33.057251 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:33.114770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0834f9-0488-4fe4-9f6d-6cd55908beb1 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:33.118318 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:33.118916 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:33.119202 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:33.119473 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:33.128289 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:33.129721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:33.139048 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1467/2910] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:28:33.559199 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:33.560724 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:33.573578 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b02e8-4d25-4748-81b2-4a0537da6df4 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:33.578714 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:33.579352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:33.579627 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:33.579892 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] PATH_INFO: `/v3/projects/0c061a803df6446a841147e2649ca62b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:33.582904 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c061a803df6446a841147e2649ca62b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:33.591650 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:33.686470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c061a803df6446a841147e2649ca62b', 'request_id': 'req-0252be6d-b868-4394-970f-5eff255f4c36'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:33.766106 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c061a803df6446a841147e2649ca62b', 'request_id': 'req-0252be6d-b868-4394-970f-5eff255f4c36'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:33.772868 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c061a803df6446a841147e2649ca62b', 'request_id': 'req-0252be6d-b868-4394-970f-5eff255f4c36'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:33.783829 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] The token cache is being invalidate because project 0c061a803df6446a841147e2649ca62b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:33.784151 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:33.784809 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:33.786290 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1444/2911] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:33 2022] DELETE /identity/v3/projects/0c061a803df6446a841147e2649ca62b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:33.790242 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:33.800346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3224c-c635-4e1b-9697-3f23739c391c service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:33.818710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:33.856520 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:33.857172 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:33.857451 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:33.857721 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] PATH_INFO: `/v3/users/e94119d1611b45c09c4953fcc92b0af4` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:33.858854 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94119d1611b45c09c4953fcc92b0af4)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:33.860825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:34.096205 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94119d1611b45c09c4953fcc92b0af4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.103095 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94119d1611b45c09c4953fcc92b0af4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.103765 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94119d1611b45c09c4953fcc92b0af4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.104407 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94119d1611b45c09c4953fcc92b0af4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.183437 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-61b2cc93-422b-4f46-a917-87b50feba8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94119d1611b45c09c4953fcc92b0af4'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.201518 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1468/2912] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:33 2022] DELETE /identity/v3/users/e94119d1611b45c09c4953fcc92b0af4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:34.630768 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:34.632265 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:34.645700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252be6d-b868-4394-970f-5eff255f4c36 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:34.703947 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:34.704618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:34.705399 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:34.705700 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] PATH_INFO: `/v3/projects/c2e05239924444beb61a5462d2c25ee1` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:34.706980 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e05239924444beb61a5462d2c25ee1)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:34.708918 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:34.804580 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:34.820657 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e05239924444beb61a5462d2c25ee1', 'request_id': 'req-4fad507b-7e4d-4536-ab0c-6641f00e88e9'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.903396 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e05239924444beb61a5462d2c25ee1', 'request_id': 'req-4fad507b-7e4d-4536-ab0c-6641f00e88e9'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.910160 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e05239924444beb61a5462d2c25ee1', 'request_id': 'req-4fad507b-7e4d-4536-ab0c-6641f00e88e9'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.921722 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] The token cache is being invalidate because project c2e05239924444beb61a5462d2c25ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:34.922006 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.922893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-4fad507b-7e4d-4536-ab0c-6641f00e88e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:34.924437 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1445/2913] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:34 2022] DELETE /identity/v3/projects/c2e05239924444beb61a5462d2c25ee1 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:34.929346 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:34.929815 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:34.929919 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:34.949140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:34.950335 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:35.008375 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1469/2914] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:35.018062 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.018352 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.018629 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.038752 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:35.039963 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:35.048045 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1446/2915] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:35.052553 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.062279 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27641-9790-4c8d-a12e-8b1560d27d8e None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.067325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.067917 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.068195 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.068502 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.069149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:28:35.069453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.071058 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.082904 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1470/2916] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 18:28:35.087073 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.096382 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455760e-de25-4d60-ae0a-6b04925b378a None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.101024 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.101615 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.101893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.102166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.102803 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.104715 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.205984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1447/2917] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:35 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:35.209941 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.219901 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5df4a-85f5-4cd5-94da-a7e87357ed2e admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.246810 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:35.277710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.278313 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.278588 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.278858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.279490 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.281396 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.386039 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1471/2918] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:35 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:35.390325 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.401594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b786-d9dd-4556-ab88-9eb2468b17f0 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.406188 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.406825 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.407105 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.407367 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.408070 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.409660 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.419478 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1448/2919] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:35.423607 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.433504 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85920e91-1495-4a0e-bc67-ce067791ad8d admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.438096 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace51ab-db7e-4e6e-af11-da9e747d4e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.438683 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace51ab-db7e-4e6e-af11-da9e747d4e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.438984 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace51ab-db7e-4e6e-af11-da9e747d4e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.439786 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace51ab-db7e-4e6e-af11-da9e747d4e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.440455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace51ab-db7e-4e6e-af11-da9e747d4e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.442009 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace51ab-db7e-4e6e-af11-da9e747d4e90 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.452527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1472/2920] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:35.457339 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.467229 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cd5c1-3dec-45a9-b52e-7b5aad2d6329 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.472086 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.472770 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.473052 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.473323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] PATH_INFO: `/v3/projects/525220d644d441a583e8902ce266a73f/users/86083ad9ea424ad6a699a0ebc617ad4c/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.488286 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525220d644d441a583e8902ce266a73f, user_id=86083ad9ea424ad6a699a0ebc617ad4c, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.491026 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.585970 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1449/2921] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:35 2022] PUT /identity/v3/projects/525220d644d441a583e8902ce266a73f/users/86083ad9ea424ad6a699a0ebc617ad4c/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:35.596182 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.596497 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.596770 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.624179 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] MFA Rules not processed for user `86083ad9ea424ad6a699a0ebc617ad4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:35.625449 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:35.687871 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1473/2922] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:35.692994 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.753933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f23f0-8220-42ce-85ad-4e8ebd77b928 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.758454 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.759049 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.759324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.759590 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.760204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.762096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:35.856031 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1450/2923] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:35 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:35.860122 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:35.871544 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513ca11-4c43-463f-af8a-285dfdbc741a None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:35.927414 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:35.928016 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:35.928293 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:35.928598 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:35.929217 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:35.931100 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:36.008887 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1474/2924] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:35 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 18:28:36.012827 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:36.022955 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef41ef2-5b70-45d7-b086-8e5efb401048 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:36.027551 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:36.028138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:36.028456 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:36.028725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:36.029363 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:36.030929 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:36.041696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1451/2925] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:36.045647 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:36.056704 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbbf5c-5dc4-4ce3-bb3b-df3913c034e8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:36.061323 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:36.061966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:36.062250 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:36.062527 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] PATH_INFO: `/v3/projects/2e06bdb133fe404f908351aaacc1318c/users/0baef6396e604e09bffcd0afa304acd5/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:36.077236 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e06bdb133fe404f908351aaacc1318c, user_id=0baef6396e604e09bffcd0afa304acd5, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:36.079853 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:36.140327 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1475/2926] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:36 2022] PUT /identity/v3/projects/2e06bdb133fe404f908351aaacc1318c/users/0baef6396e604e09bffcd0afa304acd5/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 18:28:36.144630 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:36.154732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb9d2a-4c62-4f56-96c0-b5c38e9b7448 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:36.175893 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:36.212933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b7ffe-89e3-446b-9671-4f668094c218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:36.213516 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b7ffe-89e3-446b-9671-4f668094c218 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:36.213792 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b7ffe-89e3-446b-9671-4f668094c218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:36.214058 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b7ffe-89e3-446b-9671-4f668094c218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:36.214695 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b7ffe-89e3-446b-9671-4f668094c218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:36.216271 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b7ffe-89e3-446b-9671-4f668094c218 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:36.226670 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1452/2927] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:36.230740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:36.236302 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:36.241513 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608a77-8e21-47c8-b110-255e2c1e57b9 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:36.246123 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:36.246776 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:36.247053 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:36.247344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] PATH_INFO: `/v3/projects/2e06bdb133fe404f908351aaacc1318c/users/0baef6396e604e09bffcd0afa304acd5/roles/19e3032b82e342bc93a63cb13b8b7478` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:36.249338 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e06bdb133fe404f908351aaacc1318c, user_id=0baef6396e604e09bffcd0afa304acd5, role_id=19e3032b82e342bc93a63cb13b8b7478)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:36.252085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:36.266131 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1476/2928] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:36 2022] PUT /identity/v3/projects/2e06bdb133fe404f908351aaacc1318c/users/0baef6396e604e09bffcd0afa304acd5/roles/19e3032b82e342bc93a63cb13b8b7478 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:36.273371 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:36.273656 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:36.273930 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:36.302696 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] MFA Rules not processed for user `0baef6396e604e09bffcd0afa304acd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:36.303897 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:36.371278 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1453/2929] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:36.382534 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:36.384724 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:36.443838 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0379-29b2-483a-8b45-34a1f6458ba0 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:36.449149 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:36.449801 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:36.450085 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:36.450358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:36.506884 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:36.508383 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:36.520129 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1477/2930] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:41.895227 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:41.896788 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:41.908470 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e8657-b72e-4a34-b206-4f3865336a37 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:41.911982 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb3537-c243-4fa5-80c0-dd80b99af5fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:41.912605 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb3537-c243-4fa5-80c0-dd80b99af5fe service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:41.912881 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb3537-c243-4fa5-80c0-dd80b99af5fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:41.913149 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb3537-c243-4fa5-80c0-dd80b99af5fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:41.923762 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb3537-c243-4fa5-80c0-dd80b99af5fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:41.925222 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb3537-c243-4fa5-80c0-dd80b99af5fe service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:41.934602 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1454/2931] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:28:46.513316 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:47.874762 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:47.937740 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72625c2-b1aa-46cf-9ad1-d215b517a5de service cinder] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:47.942258 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:47.942910 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:47.943184 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:47.943455 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] PATH_INFO: `/v3/users/86083ad9ea424ad6a699a0ebc617ad4c` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:47.945321 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86083ad9ea424ad6a699a0ebc617ad4c)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:47.947185 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:48.169858 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86083ad9ea424ad6a699a0ebc617ad4c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:48.176793 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86083ad9ea424ad6a699a0ebc617ad4c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:48.178594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86083ad9ea424ad6a699a0ebc617ad4c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:48.178594 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86083ad9ea424ad6a699a0ebc617ad4c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:48.265557 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86083ad9ea424ad6a699a0ebc617ad4c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:48.283531 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1478/2932] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:47 2022] DELETE /identity/v3/users/86083ad9ea424ad6a699a0ebc617ad4c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:48.288157 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:48.288675 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:48.288802 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:48.307976 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:48.309198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:48.367566 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1455/2933] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:48.377207 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:48.378585 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:48.435198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6af3dd-28c5-4f52-8648-4f98fa0fdde3 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:48.438883 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:48.439495 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:48.439775 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:48.440052 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:48.449799 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:48.451203 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:48.460640 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1479/2934] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 18:28:48.870911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:48.872405 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:48.886548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73489763-1162-4e86-a2d4-76a523b7bea8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:48.891451 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:48.892094 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:48.893083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:48.893375 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] PATH_INFO: `/v3/projects/525220d644d441a583e8902ce266a73f` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:48.894749 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=525220d644d441a583e8902ce266a73f)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:48.896732 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:49.009434 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:49.023728 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525220d644d441a583e8902ce266a73f', 'request_id': 'req-7ac4f6a6-0fc5-4595-952c-bca88e78f038'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.128156 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525220d644d441a583e8902ce266a73f', 'request_id': 'req-7ac4f6a6-0fc5-4595-952c-bca88e78f038'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.134663 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525220d644d441a583e8902ce266a73f', 'request_id': 'req-7ac4f6a6-0fc5-4595-952c-bca88e78f038'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.145776 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] The token cache is being invalidate because project 525220d644d441a583e8902ce266a73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:49.146096 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.146729 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.148546 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1456/2935] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:48 2022] DELETE /identity/v3/projects/525220d644d441a583e8902ce266a73f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:49.152747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:49.162329 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eda56a-2e97-45f5-92c3-a1f80096db61 service neutron] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:49.181325 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:49.221438 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:49.222068 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:49.222344 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:49.222616 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] PATH_INFO: `/v3/users/0baef6396e604e09bffcd0afa304acd5` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:49.223733 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0baef6396e604e09bffcd0afa304acd5)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:49.226358 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:49.422825 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0baef6396e604e09bffcd0afa304acd5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.430314 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0baef6396e604e09bffcd0afa304acd5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.430966 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0baef6396e604e09bffcd0afa304acd5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.431561 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0baef6396e604e09bffcd0afa304acd5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.497900 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-ab733132-893f-4931-857f-7a3c73703339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0baef6396e604e09bffcd0afa304acd5'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:49.515599 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1480/2936] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:28:49 2022] DELETE /identity/v3/users/0baef6396e604e09bffcd0afa304acd5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:49.970110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:49.971616 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:49.986326 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4f6a6-0fc5-4595-952c-bca88e78f038 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.044582 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.045138 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.045413 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.045682 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] PATH_INFO: `/v3/projects/2e06bdb133fe404f908351aaacc1318c` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.046791 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e06bdb133fe404f908351aaacc1318c)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.048984 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.185594 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e06bdb133fe404f908351aaacc1318c', 'request_id': 'req-2ac227dd-f036-43db-9a5e-8d36c7179dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:50.255329 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e06bdb133fe404f908351aaacc1318c', 'request_id': 'req-2ac227dd-f036-43db-9a5e-8d36c7179dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:50.261626 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e06bdb133fe404f908351aaacc1318c', 'request_id': 'req-2ac227dd-f036-43db-9a5e-8d36c7179dd0'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:50.273920 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] The token cache is being invalidate because project 2e06bdb133fe404f908351aaacc1318c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:28:50.274216 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:50.274862 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-2ac227dd-f036-43db-9a5e-8d36c7179dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:28:50.276409 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1457/2937] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:28:49 2022] DELETE /identity/v3/projects/2e06bdb133fe404f908351aaacc1318c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:50.282340 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.282807 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.282912 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.303390 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:50.305285 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:50.365885 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1481/2938] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:50.374635 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.374921 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.375198 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.394874 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:50.396204 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:50.404971 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1458/2939] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:50.409198 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:50.419441 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67231a7-4052-406b-b2d5-5f78ba0fb534 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.424297 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.424938 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.425219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.425496 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.426144 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 18:28:50.426446 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.428015 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.440326 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1482/2940] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 1 18:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 18:28:50.444812 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:50.453945 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d1cf-1795-4f54-b7d5-06545336651d None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.458585 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.459175 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.459449 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.460236 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.461110 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.462997 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.582595 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1459/2941] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 1 18:28:50 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:50.587283 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:50.597219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eec27-0747-4335-b9fd-06f819d862c7 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.623264 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:50.653672 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.654281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.654559 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.654831 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.655453 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.657366 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.789515 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1483/2942] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 1 18:28:50 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 18:28:50.793709 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:50.804324 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c13f3-578c-401f-b7a7-179bacab88f5 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.808887 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.809486 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.809786 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.810042 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.810758 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.812361 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.822973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1460/2943] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:50.827246 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:50.839276 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b53690-3596-4851-8757-5ddd5e50e3e8 admin admin] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.843847 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d44b1a-dc10-4d6f-bb24-dd630a84f784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.844483 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d44b1a-dc10-4d6f-bb24-dd630a84f784 admin admin] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.844766 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d44b1a-dc10-4d6f-bb24-dd630a84f784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.845040 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d44b1a-dc10-4d6f-bb24-dd630a84f784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.845684 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d44b1a-dc10-4d6f-bb24-dd630a84f784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.847252 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d44b1a-dc10-4d6f-bb24-dd630a84f784 admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.857650 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1484/2944] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 1 18:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 18:28:50.861908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:50.872174 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06136ae5-8abe-4dd5-8d0f-451ce1ec1014 admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:50.876711 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:50.877347 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.877624 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.877896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] PATH_INFO: `/v3/projects/c1c4aa6faa6c4fe4b9724cbe693eae17/users/247bb18b75e149e58358ae36767265b3/roles/6bc7a311754c40b2bbf21dccc998bc9b` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:50.880481 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:28:50.894511 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c4aa6faa6c4fe4b9724cbe693eae17, user_id=247bb18b75e149e58358ae36767265b3, role_id=6bc7a311754c40b2bbf21dccc998bc9b)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:50.897853 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:50.985746 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1461/2945] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 1 18:28:50 2022] PUT /identity/v3/projects/c1c4aa6faa6c4fe4b9724cbe693eae17/users/247bb18b75e149e58358ae36767265b3/roles/6bc7a311754c40b2bbf21dccc998bc9b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:28:50.990955 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e869721-1ef9-42b5-87f6-072bbb4eb47d None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:50.991360 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e869721-1ef9-42b5-87f6-072bbb4eb47d None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:50.991642 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e869721-1ef9-42b5-87f6-072bbb4eb47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:51.011188 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-8e869721-1ef9-42b5-87f6-072bbb4eb47d None None] MFA Rules not processed for user `1369adf90e6146c298833c5db7ff437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:51.012413 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-8e869721-1ef9-42b5-87f6-072bbb4eb47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:51.072223 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1485/2946] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:51.081691 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:51.083349 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:51.141781 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c0c8f-163e-421b-959e-9ab35ed42dcb admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:51.145439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:51.146038 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:51.146323 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:51.146592 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:51.155429 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:51.156908 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:51.166234 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1462/2947] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:52.480162 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bc729-7513-4ddd-941c-7027f05a68d2 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:52.480728 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bc729-7513-4ddd-941c-7027f05a68d2 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:52.480975 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bc729-7513-4ddd-941c-7027f05a68d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:52.528328 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-0c7bc729-7513-4ddd-941c-7027f05a68d2 None None] MFA Rules not processed for user `247bb18b75e149e58358ae36767265b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:52.529665 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-0c7bc729-7513-4ddd-941c-7027f05a68d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:52.596395 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1486/2948] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:52.608121 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:52.609523 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:52.665487 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77a92a-abe5-455d-b9fb-3b81270b536b service neutron] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:52.668933 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c32115f24264e738c432f700a2f7dcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:52.669535 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:52.669813 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:52.670083 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:52.678458 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:52.679878 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:52.688463 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1463/2949] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:28:54.575792 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:54.580730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:54.580730 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:54.595301 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] MFA Rules not processed for user `247bb18b75e149e58358ae36767265b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:28:54.596550 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:54.604090 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1487/2950] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:28:54.840548 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:28:54.841990 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:54.854808 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8ad15-1fdc-4956-ab9d-218d02393a82 service cinder] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:28:54.858379 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:28:54.858973 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:28:54.859254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:28:54.859528 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:28:54.861226 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:28:54.870938 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:28:54.872386 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab46656-63d4-4630-a5e8-9cdf6d8af3e6 service neutron] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:28:54.880280 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1464/2951] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:28:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:29:04.367140 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:29:04.377816 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f076-ee9f-4f9b-aaff-4c9ad408850c None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:29:04.381457 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c16d573ee440426aaa09eddb9c2658dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:29:04.382060 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:04.382339 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:04.382611 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:04.384590 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:04.416050 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:29:04.460292 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:29:04.461747 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7670892-9d08-49de-8694-c89ad2a46e2c service neutron] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:29:04.473708 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1488/2952] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:29:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:29:04.876725 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:29:13.845332 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] REQUEST_METHOD: `POST` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:13.845925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:13.845925 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:13.864639 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.auth.core [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] MFA Rules not processed for user `247bb18b75e149e58358ae36767265b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:29:13.865834 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:13.872767 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1465/2953] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 1 18:29:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:29:14.522781 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=20 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:29:23.859317 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=3 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:29:38.534551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e1f45-d2c3-4a9b-9bd0-388deb8b48eb None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:38.534551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e1f45-d2c3-4a9b-9bd0-388deb8b48eb None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:38.534551 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e1f45-d2c3-4a9b-9bd0-388deb8b48eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:38.594509 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-a00e1f45-d2c3-4a9b-9bd0-388deb8b48eb None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:29:38.595710 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-a00e1f45-d2c3-4a9b-9bd0-388deb8b48eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:38.658864 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1489/2954] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 1 18:29:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:29:38.799529 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:29:38.800958 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:38.894254 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98446-a784-41ad-a02d-e2948b4dc4f8 None None] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:29:38.899241 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:29:38.899842 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:38.900124 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:38.900427 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:38.909067 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:29:38.910647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:29:38.918647 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1466/2955] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:29:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 18:29:48.656019 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=44 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:29:48.915539 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=41 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:29:55.096593 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] REQUEST_METHOD: `POST` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:55.097099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:55.097212 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:55.116545 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.auth.core [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] MFA Rules not processed for user `16dc787e53484e0ea5ab6757d99c8a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 18:29:55.117769 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:55.125102 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1490/2956] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 1 18:29:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 18:29:55.137342 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:29:55.148469 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f48f81-8adf-408f-9a37-65e872b97f50 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:29:55.153439 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42c1211f644d45bb8979455c4f28284a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:29:55.154028 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:55.154308 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:55.154584 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:55.163300 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:29:55.164905 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:29:55.172911 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1467/2957] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 1 18:29:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 18:29:57.946235 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:29:57.947803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:58.017281 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056544b9-d829-4158-9a76-4754ccf5ea02 None None] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:29:58.022219 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:29:58.022824 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] REQUEST_METHOD: `GET` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:58.023099 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:58.023369 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:58.025125 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.fernet_utils [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:29:58.045164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:29:58.045164 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:29:58.047426 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1491/2958] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:29:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:29:58.088591 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:29:58.089423 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0840-9721-4bf5-b36d-d9ae745580d7 service swift] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:29:58.094252 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:29:58.094884 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:58.095176 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:58.095520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] PATH_INFO: `/v3/users/247bb18b75e149e58358ae36767265b3` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:58.096760 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=247bb18b75e149e58358ae36767265b3)` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:29:58.098805 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:29:58.290506 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247bb18b75e149e58358ae36767265b3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.297618 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247bb18b75e149e58358ae36767265b3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.373552 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247bb18b75e149e58358ae36767265b3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.374166 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247bb18b75e149e58358ae36767265b3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.386986 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.notifications [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247bb18b75e149e58358ae36767265b3'} {{(pid=81703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.393012 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1468/2959] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 1 18:29:58 2022] DELETE /identity/v3/users/247bb18b75e149e58358ae36767265b3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:29:58.678829 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] Authenticating user token {{(pid=81704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:29:58.691604 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d0fa8-87ed-47f2-860a-25ebadd2b4a3 service placement] Validating token access rules against request {{(pid=81704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:29:58.750389 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1369adf90e6146c298833c5db7ff437b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '195190be58ff49f7a586c1baca5891d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:29:58.751021 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:29:58.751300 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:29:58.751573 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] PATH_INFO: `/v3/projects/c1c4aa6faa6c4fe4b9724cbe693eae17` {{(pid=81704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:29:58.752803 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c4aa6faa6c4fe4b9724cbe693eae17)` {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:29:58.754674 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] RBAC: Authorization granted {{(pid=81704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:29:58.848444 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c4aa6faa6c4fe4b9724cbe693eae17', 'request_id': 'req-0bfd2e39-634c-4a77-92bf-f6286c087f9c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.910890 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c4aa6faa6c4fe4b9724cbe693eae17', 'request_id': 'req-0bfd2e39-634c-4a77-92bf-f6286c087f9c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.917079 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c4aa6faa6c4fe4b9724cbe693eae17', 'request_id': 'req-0bfd2e39-634c-4a77-92bf-f6286c087f9c'} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.929027 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] The token cache is being invalidate because project c1c4aa6faa6c4fe4b9724cbe693eae17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81704) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 18:29:58.929249 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.929877 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG keystone.notifications [None req-0bfd2e39-634c-4a77-92bf-f6286c087f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81704) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 18:29:58.931372 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: [pid: 81704|app: 0|req: 1492/2960] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 1 18:29:58 2022] DELETE /identity/v3/projects/c1c4aa6faa6c4fe4b9724cbe693eae17 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 18:30:08.389416 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:30:08.925897 dv-cinder-749155-10-iscsi devstack@keystone.service[81704]: DEBUG dbcounter [-] [81704] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 18:33:20.385336 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Authenticating user token {{(pid=81703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 18:33:20.386738 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:33:20.524050 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16489f4-7a95-453e-9930-87c15af6a0db admin admin] Validating token access rules against request {{(pid=81703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 18:33:20.582769 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c88695a14384a0b85bb20a0589692bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcfaa7697faf49c29c93c4524c09369f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 18:33:20.583402 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] REQUEST_METHOD: `GET` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 18:33:20.583665 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] SCRIPT_NAME: `/identity` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 18:33:20.583966 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 18:33:20.585896 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.fernet_utils [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 18:33:20.599520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 18:33:20.601143 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20377fc-b356-40c7-a89a-33e6548fbd1c service placement] RBAC: Authorization granted {{(pid=81703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 18:33:20.611818 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: [pid: 81703|app: 0|req: 1469/2961] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 1 18:33:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 18:33:30.607520 dv-cinder-749155-10-iscsi devstack@keystone.service[81703]: DEBUG dbcounter [-] [81703] Writing DB stats keystone:SELECT=89 {{(pid=81703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}